Apache Solr Release Notes

This file lists Solr's raw release notes with details of every change to Solr.
Most people will find the solr-upgrade-notes.adoc file more approachable.
https://github.com/apache/solr/blob/main/solr/solr-ref-guide/modules/upgrade-
notes/pages/solr-upgrade-notes.adoc

================== 9.2.0 ==================

New Features
---------------------

* SOLR-16435: Add Request timeout to Http2SolrClient (Tomás Fernández Löbbe)

* SOLR-16500: Added --enable-preview for JDK 19 to enable MemorySegment support in

MMapDirectory (Kevin Risden)

* SOLR-16496: QueryElevationComponent now supports fq exclusions (Rudi Seitz via

David Smiley)

* SOLR-16409: Admin UI - Expose all highlighting parameters in the Query UI (Jeanie

Lam via Eric Pugh)

* SOLR-16532: New OpenTelemetry (OTEL) module with OTLP/gRPC trace exporter. See
ref.guide. (janhoy, David Smiley)

* SOLR-16596: Learning To Rank - Added support for null feature values in multiple
additive trees models (Anna Ruggero via Alessandro Benedetti)

* SOLR-16608: Ability to compress state.json in Zookeeper (Justin Sweeney via

noble)

* SOLR-16646: New function query operator isnan to verify if value is NaN (Gabriel
Magno via Kevin Risden)

* SOLR-16643: Add reRankOperator=multiply/replace options to rerank query parser

(Andy Webb, Christine Poerschke, Mikhail Khludnev)

Improvements
---------------------

* SOLR-16428: IgnoreLargeDocumentProcessorFactory now supports a "permissive" mode,

where it logs and skips
offending documents but doesn't short-circuit the entire batch or return a 4xx
error. This mode can be enabled
by setting the `permissiveMode` boolean option to `true` in your solrconfig.xml's
IgnoreLargeDocumentProcessorFactory declaration. (Jason Gerlowski)

* SOLR-16392: A number of v2 "replica" endpoints have been updated to be more REST-

ful, and hopefully, intuitive.
ADDREPLICAPROP is now `PUT
/api/collections/$coll/shards/$shard/replicas/$rep/properties/$prop {"value":
$val}`
DELETEREPLICAPROP is now `DELETE
/api/collections/$coll/shards/$shard/replicas/$rep/properties/$prop`

* SOLR-11657: Deprecate ContentStream in favour of RequestWriter.ContentWriter.

(Joshua Ouma via Eric Pugh)
* SOLR-16302: WARN when restoring backup if ConfigSet with same name already exists
(Albert Moser via Kevin Risden)

* SOLR-8803: Java will now crash the Solr process on OOME and create a crash file
that logs the
cause. This capability is now also present on Windows. Before, with bin/solr
but not
bin/solr.cmd, Solr would execute a script that killed the Solr pid, leaving a
very small time
window where Solr would continue to execute in an unpredictable state. (Shawn
Heisey, Kevin Risden)

* SOLR-15748: A v2 equivalent of CLUSTERSTATUS command is now available at `GET

/api/cluster`. Collection listing,
previously at this path, can still be accessed at `GET /api/collections`. (Joshua
Ouma via Jason Gerlowski)

* SOLR-16475: Make the default replica placement plugin configurable as system

property on startup (janhoy)

* SOLR-13626: Document the SystemInfoHandler in the Ref Guide. (Tony Cook via Eric
Pugh)

* SOLR-16420: Introducing `{!mlt_content}foo bar` to cover existing `/mlt` handler

functionality for SolrCloud.
(Mikhail Khludnev)

* SOLR-11028: A v2 equivalent of the `/admin/collections?action= REPLACE` command

is now available at
`POST /api/cluster/nodes/nodeName/replace`. (Joshua Ouma via Jason Gerlowski)

* SOLR-15479: A v2 equivalent of the RENAME command is now available at `POST

/api/collections/collName/rename`. (Anakhe Ajayi via Jason Gerlowski)

* SOLR-16158: Certain Jetty "server" jars (jetty-util, jetty-io, etc.) are no

longer duplicated between server/lib and WEB-INF/lib. All shared jars now live in
server/lib/ext.
By default, Jetty does not allow these jars to be shared by the Jetty server and
the web-app (Solr).
In order to accomplish this, explicit exclusions have been added to
server/contexts/solr-jetty-context.xml that allow these "server" jars to be shared.
Solr Core also no longer contains the JettySolrRunner (now living in the testing
framework), removing the need for Jetty Server dependencies in solr-core.
(Houston Putman)

* SOLR-16565: posting the same file to the package store should not throw an error
(noble)

* SOLR-8975: Introduce Builder setters for setters that exist on SolrClients for
responseParser, requestWriter.
Deprecated these SolrClient setters useMultiPartPost, and followRedirects. (Eric
Pugh, David Smiley, Kevin Risden)

* SOLR-15787: FileSystemConfigSetService: implement the abstraction completely. It

could be useful
for putting ConfigSets on a shared file system. (Nazerke Seidan, David Smiley)

* SOLR-16438: Support optional split.setPreferredLeaders prop in shard split

command. (Bruno Roustant)
* SOLR-10463: Introduce Builder setter for retryExpiryTime on cloud SolrClients.
Deprecated
direct setter setRetryExpiryTime on cloud SolrClients. (Eric Pugh)

* SOLR-10461: Introduce Builder setter for aliveCheckInterval on load balanced

SolrClients. Deprecated
direct setter setAliveCheckInterval on SolrClients. (Eric Pugh, David Smiley,
Alex Deparvu)

* SOLR-10462: Introduce Builder setter for pollQueueTime on

ConcurrentUpdateHttp2SolrClient. Deprecated
direct setter setPollQueueTime on ConcurrentUpdateHttp2SolrClient. (Eric Pugh)

* SOLR-10464: Introduce Builder setter for collectionCacheTtl on cloud SolrClients.

Deprecated
direct setter setCollectionCacheTTL on cloud SolrClients. (Eric Pugh, David
Smiley)

* SOLR-10452: Introduce Builder setter withTheseParamNamesInTheUrl for queryParams,

renaming them to urlParamNames
to clarify they are parameter names, not the values. Deprecated direct setter
setQueryParams and addQueryParams
on SolrClients. (Eric Pugh, David Smiley, Alex Deparvu)

* SOLR-10470: Introduce Builder setter for parallelCacheRefreshes on cloud

SolrClients. Deprecated
direct setter setParallelCacheRefreshes on cloud SolrClients. (Eric Pugh, David
Smiley, Alex Deparvu)

* SOLR-16616: JWTAuthPlugin: Read trusted X509 certificates from multiple files

(janhoy)

* SOLR-15772: More visible security warnings in Admin UI (janhoy)

* SOLR-15787: FileSystemConfigSetService: implement the abstraction completely. It

could be useful
for putting ConfigSets on a shared file system. (Nazerke Seidan, David Smiley)

* SOLR-6312: SolrJ Cloud clients now correctly support isUpdatesToLeaders() being

false. This behavior can also be
configured per AbstractUpdateRequest instance. (hossman)

* SOLR-16635: Resolve crypto keys file path in MiniSolrCloudCluster. This

eliminates the need to set the system
properties `pkiHandlerPublicKeyPath` and `pkiHandlerPrivateKeyPath` before
starting MiniSolrCloudCluster when using
the Solr test framework. (Tomás Fernández Löbbe)

* SOLR-16618: Admin UI Analysis page should include dynamic fields (Alex Deparvu
via Eric Pugh)

* SOLR-16590: Standardize Builder method names on SolrClient's to use the with

pattern, deprecating the use of set or bare property name. (Eric Pugh)

* SOLR-16595: Standardize Builder methods handling of times to use require a

TimeUnit to be passed in. Deprecated methods that
do not specify a TimeUnit. (Eric Pugh)
* SOLR-16665: The base docker image has been upgraded from Ubuntu 20 (Focal) to
Ubuntu 22 (Jammy). (Houston Putman)

* SOLR-11029: A v2 equivalent of the `/admin/collections?action=DELETENODE` command

is now available at
`POST /api/cluster/nodes/nodeName/clear`. (Bence Szabo via Jason Gerlowski)

* SOLR-16665: The base docker image has been upgraded from Ubuntu 20 (Focal) to
Ubuntu 22 (Jammy). (Houston Putman)

* SOLR-16462: v2 equivalents of the "Core Admin" `LISTSNAPSHOT`, `CREATESNAPSHOT`,

and `DELETESNAPSHOT` commands are now available at
`GET /api/cores/coreName/snapshots`, `POST
/api/cores/coreName/snapshots/snapshotName`, and
`DELETE /api/cores/coreName/snapshots/snapshotName`, respectively (John Durham
via Jason Gerlowski)

* SOLR-16488: The paths of the v2 "ZooKeeper read" APIs have been tweaked slightly
to be more intuitive. "get node data" is now available
at `GET /api/cluster/zookeeper/data/<path>`, and "list node children" is now
available at `GET /api/cluster/zookeeper/children/<path>`
(Jason Gerlowski, Joshua Ouma)

* SOLR-16393: The path of the v2 "list alias" API has been tweaked slightly to be
more intuitive, and is now available at
`GET /api/aliases`. It is also now possible to request information about a
specific alias at `GET /api/aliases/<aliasName>`.
(Alex Deparvu via Jason Gerlowski)

* SOLR-16397: /mlt now has a v2 API available at `GET

/api/collections/collName/mlt` (Ameer Albahem via Jason Gerlowski)

Optimizations
---------------------

* SOLR-16515: Remove synchronized access to cachedOrdMaps in

SlowCompositeReaderWrapper (Dennis Berger, Torsten Bøgh Köster, Marco Petris)

* SOLR-16555: SolrIndexSearcher - FilterCache intersections/andNot should not clone

bitsets repeatedly (Kevin Risden, David Smiley)

* SOLR-15732: queries to missing collection are slow (noble)

* SOLR-15616: Allow thread metrics to be cached (Ishan Chattopadhyaya, ab)

* SOLR-16615: Jersey 'ApplicationHandlers' are now shared by compatible cores where

possible (Jason Gerlowski, Houston Putman)

* SOLR-16487: Replication pooling is optimized based on hard and soft commit

settings for a given collection (Justin Sweeney)

* SOLR-16689: Avoiding commits on leader when recovering a non-leader replica and

avoiding incorrectly replicating an empty core (Justin Sweeney)

Bug Fixes
---------------------

* SOLR-16436: Fix "false positive" suggestions from DirectSolrSpellChecker when

using maxQueryFrequency > 1 in
 multi-shard collections (hossman)

* SOLR-15854: Upgrade semver library to fix versioning constraint bugs in the

PackageManager. (Houston Putman, Kevin Risden)

* SOLR-16160: UpdateXmlMessages duplicate data when data is removed and then added
in the same message (Alex Deparvu via Kevin Risden)

* SOLR-16483: Fix IndexOutOfBounds in RecursiveNumericEvaluator (Bendegúz Ács via

Kevin Risden)

* SOLR-16363: DirectUpdateHandler2 should not throw

UnknownFormatConversionException (Kevin Risden)

* SOLR-16293: Luke request fails for document with a binary field (Kevin Risden)

* SOLR-16477: Collection RENAME api creates broken alias (Alex Deparvu via Kevin
Risden)

* SOLR-10918: Fix IntPointField hashing for HLL (Houston Putman)

* SOLR-16274: HEAD request for managed resource returns 500 Server Error (Kevin
Risden)

* SOLR-16528: Jaegertracer module must include okhttp3 dependency (janhoy)

* SOLR-16420: Default for cloud mode was fixed to `{!mlt mindf=5}` to comply with
Reference Guide (Mikhail Khludnev)

* SOLR-16473: Fix race condition in shard split when a sub-shard is put in recovery
state. (Andy Vuong via Bruno Roustant)

* SOLR-10458: Fix followRedirect property on HttpSolrClient not set when using

Builder pattern. (Eric Pugh)

* SOLR-16605: CPU percent calculation incorrect in admin UI cloud->nodes tab (Shawn

Heisey)

* SOLR-16611: NullPointerException occus when there are no segments in `{!collapse

hint=top_fc}` (Minami Takuya via Mikhail Khludnev)

* SOLR-16613: CryptoKeys should handle RSA padding for OpenJ9 (Kevin Risden)

* SOLR-16621: Admin UI fails to grant user permissions that have wildcard role
(janhoy)

* SOLR-16631: solr.allowUrls (former solr.shardsWhitelist) should treat hostnames

in case insensitive way. (Paul Blanchaert via Eric Pugh)

* SOLR-16647: Fix circuit breaker examples in solrconfig.xml (Colvin Cowie via

Kevin Risden)

* SOLR-16648: NullPointerException when excluding facets in More Like This Handler

(Mikhail Khludnev)

* SOLR-16628: Ensure that InputStreams are closed after Xml parsing (Michael
Gibney, David Smiley, Kevin Risden)

* SOLR-16639: Fix jq parse error of solr-exporter metrics

node_thread_pool_completed_total (Naoto Minami)

* SOLR-16658: List of permissions returned to Admin UI is not complete (janhoy)

* SOLR-9698: Fix start/stop wait time and RMI_PORT on Windows (Colvin Cowie)

* SOLR-16670: Fix directory/file check in S3Repository (Houston Putman, Hakan

Özler)

* SOLR-16668: Use default to Java SSL for Http2SolrClient when none is provided
(Houston Putman)

* SOLR-16669: Http2SolrClient now defaults checkPeerName to True, as the

documentation specified (Houston Putman)

* SOLR-16679: Fix solr.jetty.ssl.verifyClientHostName logging (Kevin Risden)

* SOLR-16682: MoreLikeThis Component fails with SyntaxError: Cannot parse if

document terms contains symbols from query parser syntax
(Mikhail Khludnev)

* SOLR-16656: rid parameter missing from query logs (Alex Deparvu)

* SOLR-16676: Logs: Http2SolrClient.async() lacked MDC (Alex Deparvu)

* SOLR-16686: Using bin/solr to copy a file from ZK to local fails if the local
filename does not have a path.
(Shawn Heisey)

* SOLR-16653 : sub-shard remains inactive after split for NRT+PULL collection

(Hitesh Khamesra, noble)

Build
---------------------
* SOLR-16476: Remove commons-text dependency from solr-core (Kevin Risden)

* SOLR-16624: Remove Gradle Groovy version override (Kevin Risden)

* SOLR-16641: Generate gradle.properties from gradlew, if absent (Colvin Cowie)

Dependency Upgrades
---------------------
* SOLR-15955: Upgrade to Jetty 10.x (Mark Miller, Kevin Risden)

* PR#1052: Upgrade forbiddenapis to 3.4 (Uwe Schindler)

* SOLR-16432, SOLR-16508: Upgrade the gradle wrapper to 7.6 (Shawn Heisey, Kevin
Risden)

* SOLR-16442: Upgrade to Lucene 9.4.2 (Christine Poerschke)

* SOLR-16578: Upgrade to errorprone 2.18.0 (Kevin Risden)

* SOLR-16625: Upgrade OWASP dependency check to 8.0.1 (Kevin Risden)

* SOLR-16545: Upgrade Carrot2 to 4.5.0 (Dawid Weiss)

* SOLR-16481: Upgrade Bats to 1.8.0 (Kevin Risden)

* SOLR-16467, PR#1426: Upgrade semver4j to v4, fixes failures with some locales
(Kevin Risden)

* SOLR-16529: Upgrade jaeger-client to 1.8.1 (janhoy)

* SOLR-16579, PR#1379: Upgrade Jackson to 2.14.2 (Kevin Risden)

* SOLR-16562, PR#1433: Upgrade to Caffeine 3.1.5 (Kevin Risden)

* SOLR-16626, PR#1392: Upgrade to Netty v4.1.89.Final (Kevin Risden)

* SOLR-16627, PR#1399, PR#1401: Upgrade google-cloud-bom to v0.190.0, re2j to 1.7,

and grpc to 1.53.0 (Kevin Risden)

* PR#1425: Update dependency org.jctools:jctools-core to v4 (solrbot)

* PR#1421: Update org.apache.logging.log4j:* to v2.20.0 (solrbot)

* PR#1420: Update org.apache.calcite:* to v1.33.0 and org.apache.calcite.avatica:*

to v1.23.0 (solrbot)

* PR#1419: Update io.prometheus:* to v0.16.0 (solrbot)

* PR#1417: Update dependency org.openjdk.jmh:jmh-core to v1.36 (solrbot)

* PR#1416: Update dependency org.bitbucket.b_c:jose4j to v0.9.3 (solrbot)

* PR#1415: Update dependency org.apache.commons:commons-compress to v1.22 (solrbot)

* PR#1406: Update dependency net.sourceforge.argparse4j:argparse4j to v0.9.0

(solrbot)

* PR#1404: Update dependency joda-time:joda-time to v2.12.2 (solrbot)

* PR#1403: Update dependency commons-cli:commons-cli to v1.5.0 (solrbot)

* PR#1402: Update dependency com.tdunning:t-digest to v3.3 (solrbot)

* PR#1398: Update dependency com.fasterxml.woodstox:woodstox-core to v6.5.0

(solrbot)

* PR#1397: Update dependency biz.aQute.bnd:biz.aQute.bnd.annotation to v6.4.0

(solrbot)

* PR#1396: Update org.slf4j:* to v2.0.6 (solrbot)

* PR#1395: Update org.apache.zookeeper:* to v3.8.1 (solrbot)

* PR#1394: Update org.apache.tika:* to v1.28.5 (solrbot)

* PR#1393: Update org.apache.httpcomponents. httpclient v4.5.14, httpcore v4.4.16,

httpmime v4.5.14 (solrbot)

* PR#1391: Update io.dropwizard.metrics:* to v4.2.17 (solrbot)

* PR#1390: Update dependency org.xerial.snappy:snappy-java to v1.1.9.1 (solrbot)

* PR#1388: Update dependency org.immutables:value-annotations to v2.9.3 (solrbot)

* PR#1387: Update dependency net.thisptr:jackson-jq to v0.0.13 (solrbot)

* PR#1383: Update dependency io.swagger.core.v3:swagger-annotations to v2.2.8

(solrbot)

* PR#1382: Update dependency com.lmax:disruptor to v3.4.4 (solrbot)

* PR#1381: Update dependency com.github.spotbugs:spotbugs-annotations to v4.7.3

(solrbot)

Other Changes
---------------------
* SOLR-16150: Embedded ZK Server used for zkRun should advertise itself on specific
bound interfaces instead of
generically on localhost. (Mike Drob)

* SOLR-15718: Remove backcompat feature solr.useUnsafeOverseerResponse (janhoy)

* SOLR-16541: Eliminate the 'numShards' system property (noble)

* SOLR-15861: ConcurrentUpdateSolrClient should work with ManagedExecutorService

(Sammy Chu, Kevin Risden)

* SOLR-16575: splitshard should honour createNodeSet (noble)

* SOLR-16569: Add java system property to overseer queue size (Nick Ginther via
noble)

* SOLR-16574: Demonstrate Dense Vectors and KNN as part of the Films example
(Gabriel Magno via Eric Pugh)

* SOLR-16534: Jaegertracer-Configurator is now deprecated. From v10.0 we'll only

support OpenTelemetry (janhoy)

* SOLR-14336: Warning about potentially old Solr version in Admin UI dashboard

(janhoy)

* SOLR-13243: Correct the initial capacity of the ZK operations to run in

ShardLeaderElectionContextBase#runLeaderProcess (Haythem Khiri)

* SOLR-16544: Improve documentation on how to contribute to Solr (Justin Sweeney)

* SOLR-16591: The "Transient Cores" feature is now deprecated. In solr.xml

"<transientCoreCacheFactory>"
no longer works; it wasn't documented either. Some internals were changed as
well to simplify
the standard case of having no transient cores. (David Smiley)

* SOLR-16629: Replace Solr thread safety annotations with JCIP. (David Smiley)

* SOLR-16632: Add core name to periodic delete related log messages (Bence Szabo
via Andras Salamon)

* SOLR-15959: Deprecate loading of solr.xml from Zookeeper (janhoy)

* SOLR-16573: Introduce EmbeddedSolrServerTestRule, a JUnit 4 TestRule for Solr

testing
(Joshua Ouma, David Smiley)
* SOLR-16660: Deprecate analytics component (janhoy)

* SOLR-15733: Separate out a solrj-streaming module (Joel Bernstein, Kevin Risden)

* SOLR-15928: Dim 'Add Collection' and 'Create Alias' buttons when user lacks
proper permissions (janhoy)

* SOLR-16672: Simplify SolrCLI Healthcheck command's logic for obtaining a target

collection (Eric Pugh)

* SOLR-16684: Keep solr's opennlp-tools version in sync with Lucene (janhoy)

* SOLR-16681: Throw exception when attempting to replace uniqueKey via fl in

distributed request,
e.g. fl=old_id:id,id:new_id (Mikhail Khludnev)

================== 9.1.1 ==================

Bug Fixes
---------------------

* SOLR-16589: Large fields with large=true can be truncated when using unicode
values (Kevin Risden)

* SOLR-16585: Fixed NPE when paginating MatchAllDocs with non-zero start offset,
like q=*:*&start=10 (Michael Gibney)

* SOLR-16165: Rare Deadlock in SlotAcc initialization (Justin Sweeney, noble)

* SOLR-16622: Replicas don't come up active after node restart (noble, Ishan
Chattopadhyaya)

Other Changes
---------------------

* SOLR-16443: Upgrade Jackson bom to 2.13.4.20221013 (Kevin Risden)

* SOLR-16568: Upgrade woodstox-core to 6.4.0 (Kira Traynor)

* SOLR-16598: Upgrade Protobuf to 3.21.12 (David Smiley)

* SOLR-16567: Fixed problem with filtering and KNN search, especially when using
post-filters (Alessandro Benedetti)

* SOLR-16588: Fixed problem with default knn algorithm (Elia Porciani via
Alessandro Benedetti)

* SOLR-16480: ConfigSets now have an overridable allow-list for filetypes. (Houston

Putman)

================== 9.1.0 ==================

New Features
---------------------
* SOLR-16246: Introduced pre-filtering in KnnQParser (Elia Porciani via Alessandro
Benedetti).

* SOLR-16111: Add hl.queryFieldPattern support, an advanced alternative to the

hl.requireFieldMatch boolean flag.
 (Christine Poerschke, David Smiley)

* SOLR-15921: Load jars in <solr-install-dir>/lib/ by default (janhoy)

* SOLR-16005: Support add/update/delete field types in Admin UI Schema page.

(Ahmet Can Kepenek, Eric Pugh)

* SOLR-15853: Admin UI support for managing Paramsets and using in Queries. (Betul
Ince, Eric Pugh)

* SOLR-16096: Support createNodeSet parameter when creating collections in Admin

UI. (Eric Pugh)

* SOLR-14319: Add ability to specify replica types when creating collections in

Admin UI. (Richard Goodman, Eric Pugh)

* SOLR-16282: CoreAdminHandler supports custom actions via solr.xml configuration.

(Artem Abeleshev, Christine Poerschke)

* SOLR-15007: Add ability to roll up core level metrics to be node level metrics
for a RequestHandler via configuration. (Justin Sweeney, David Smiley)

* SOLR-15715: Dedicated query coordinator nodes in the solr cluster (noble, Hitesh
Khamesra, Ishan Chattopadhyaya)

Improvements
---------------------
* SOLR-15986: CommitUpdateCommand and SplitIndexCommand can write user commit
metadata. (Bruno Roustant)

* SOLR-15652: Add Slack channel for community support to Admin UI footer, update
links. (Eric Pugh)

* SOLR-16002: Avoid redundant `FilterQuery` caching, e.g. via `filter($query)`

syntax. (Michael Gibney, David Smiley)

* SOLR-15045: `DistributedZkUpdateProcessor` now issues commits to local shards and

remote shards in parallel,
halving the latency of synchronous commits (Michael Gibney)

* SOLR-16181: Initialize the LogWatcher earlier in CoreContainer#load() (janhoy)

* SOLR-16192: Add ZK credentials injectors support (Lamine Idjeraoui with review by

Anshum Gupta, Jason Gerlowski, Mike Drob and Houston Putman)

* SOLR-16225: Upgrade dependencies (Carrot2, HPPC) (Dawid Weiss)

* SOLR-16257: Improve ZkStateReader to avoid race condition between

collectionWatches and watchedCollectionStates
(Patson Luk, Houston Putman, Mike Drob)

* SOLR-16323: The Docker image now uses the Solr User ID instead of the User Name,
helps with non-root checks (Houston Putman)

* SOLR-16340: Provide a cluster_id label for all metrics exposed by the Solr
Prometheus Exporter, making it possible to
filter Grafana board by cluster (janhoy, Houston Putman)

* SOLR-16337: implement Zk metrics (noble)

* SOLR-16353: Make SplitShardCmd#checkDiskSpace disableable through a system
property. (Haythem Khiri)

* SOLR-16362: Logs: Truncate field values in logs if a doc fails to index. (Nazerke
Seidan, David Smiley)

* SOLR-16230: JWT nested roles support (Marco Descher, janhoy)

* SOLR-16361: mod() is now accurate for all integers, floats, doubles and longs
upto 2^52 (Dan Rosher via Eric Pugh)

* SOLR-16404: Prometheus Exporter: Use HTTP2 Solr clients. (Houston Putman)

* SOLR-16429: The s3-repository module now supports AWS STS, and therefore Web
Identity Tokens, by default. (Josh Souza via Houston Putman)

Optimizations
---------------------
* SOLR-16120: Optimise hl.fl expansion. (Christine Poerschke, David Smiley, Mike
Drob)

* SOLR-14765: Optimize DocList creation for sort-irrelevant cases. This issue also
reworks the building and caching of liveDocs
in SolrIndexSearcher, and refines/clarifies the effect of
`useFilterForSortedQuery` (Michael Gibney, Mike Drob, David Smiley)

* SOLR-16146: Avoid loading all collections during node startup (noble)

* SOLR-16266: Eliminate unneccessary byte[] copy in Http2SolrClient (hossman)

* SOLR-9359: Enable warming queries to be managed using Config API (Andy Webb, Eric
Pugh)

* SOLR-16336: avoid fetching solrconfig.xml & schema.xml for already cached schema
and config (noble)

* SOLR-16328: intern() strings in DocCollection to reduce memory footprint (noble)

* SOLR-16445: Leader message should only be sent when there is more than one
(NRT+Tlog) replica (Hitesh Khamesra, noble via Ishan Chattopadhyaya)

Bug Fixes
---------------------
* SOLR-13219: Fix NPE in FieldLengthFeature with non-stored/missing fields (Nick
Veenhof, Tomasz Elendt)

* SOLR-15918: Skip repetitive parent znode creation on config set upload (Mike
Drob)

* SOLR-15964: Transient cores: don't evict a core when it's still being used.
(David Smiley)

* SOLR-15116: Support a HEAD request for managed resources. (Eric Pugh)

* SOLR-16089: Fix empty columns in Cloud Node UI Screen when replica is in down
state. (hossman)

* SOLR-16112: DefaultSolrHighlighter.doHighlighting to Query#rewrite multiple times

if necessary. (Christine Poerschke)

* SOLR-13044: Fix NPE during core close racing with index replication check.
(David Smiley, hossman)

* SOLR-16145: Fix very rare NPE in SolrCloud rejoinOverseerElection. (David Smiley)

* SOLR-16168: Spellcheck NPE if invalid dictionary name is provided (Kevin Risden)

* SOLR-15830: Concurrent core reloads mess up commits when using Schema API (Bence
Szabo via Andras Salamon)

* SOLR-16169: IndexBasedSpellChecker with empty spellcheck.q results in

NegativeArraySizeException (Kevin Risden)

* SOLR-16110: Using Schema/Config API breaks the File-Upload of Config Set File
(Steffen Moldenhauer, Kevin Risden)

* SOLR-16218: failOnVersionConflicts option not working for in-place updates

(Lamine Idjeraoui)

* SOLR-16259: Missing newline at the end of solr.in.sh breaks the installer. (John
Gately via Shawn Heisey)

* SOLR-16241: Fix JettyConfig.builder(JettyConfig) to correctly copy all attributes

(hossman)

* SOLR-16142: Fix Admin UI's spatial parameter generation. (Arsal Jalib, Christine
Poerschke)

* SOLR-9661: Fix explanation of select() streaming expression that uses replace()

operation (Ahmet Can Kepenek via Eric Pugh)

* SOLR-16316: Fix debugQuery clicking on the Query UI does not show the
debug.explain.structured option. (Shiming Li via Eric Pugh)

* SOLR-16399: ExportWriter fails with max values for fields (Kevin Risden)

* SOLR-16357: CaffeineCache ramBytes incorrect accounting on updates and deletes

(Alex Deparvu)

* SOLR-16343: Handle MDC snapshot being null (Dee Moore via Kevin Risden)

* SOLR-7883: MoreLikeThisHandler to support facets (Mikhail Khludnev)

* SOLR-16406: Take SOLR_MODULES, configured by solr.in.sh, into account. (Thomas

Wöckinger via janhoy)

* SOLR-16344: PlacementPlugin throws NPE for PRS collections (noble)

* SOLR-16417: NPE if facet query hits timeout or exception (Kevin Risden)

* SOLR-16414: Race condition in PRS state updates (noble, Justin Sweeney, Patson
Luk, Hitesh Khamesra, Ishan Chattopadhyaya)

* SOLR-16418: Introduce SolrResponseUtil to handle NPE during query timeout or

exception when parsing SolrResponse (Kevin Risden)

* SOLR-16219: ICUCollationField protected field IllegalAccessException from

different classloader (Michael Gibney)

* SOLR-16426: LBHttp2SolrClient (and therefore CloudHttp2SolrClient) now respect

requestWriter, responseParser and queryParam changes (Houston Putman)

* SOLR-16232: Fix EnvVar usage in bin/solr (Houston Putman)

* SOLR-16439: Cloud Node does not have Core, the admin UI cannot be displayed
properly. (Shiming Li)

* SOLR-16229: Http2SolrClient aborts requests on exception (Daniel Rabus, Tomás

Fernández Löbbe)

* SOLR-16457: solr.data.home should not be set to empty string in bin/solr (Kevin

Risden)

* SOLR-16433: Security Manager prevents Solr SQL from working (Kevin Risden)

* SOLR-16460: ClusterState.copyWith is inconsistent (noble)

* SOLR-16440: RefreshCollectionMessage in PRS should update Overseer's

ZkStateWriter (noble, Ishan Chattopadhyaya)

* SOLR-16453: Overseer doesn't handle PRS and non-PRS messages properly (Justin
Sweeney, Hitesh Khamesra)

* SOLR-16463: Workaround for serious crash on JDK17+ due to JIT on caffeinecache

(janhoy, Uwe Schindler)

* SOLR-16478 :removing a replica should delete the PRS entry from the hostnode and
not from overseer (Patson Luk via noble)

* SOLR-16485: Fix NPE in ShardHandlerFactory when running in Standalone mode

(Houston Putman)

* SOLR-16412: Race condition could trigger error on concurrent

SizeLimitedDistributedMap cleanup (Patson Luk via noble)

* SOLR-16416: OverseerPrioritizer now runs after all handlers are registered, and
retries on failures. (Houston Putman)

* SOLR-14679: Fix continuously growing TLOGs on TLOG replicas (Viktor Molnár via
Houston Putman)

* SOLR-16502: Multiple CopyField should not limit to first maxChars (Fredrik

Rodland, Kevin Risden)

* SOLR-16527: RuleBasedAuthorizationPluginBase NPE (Alex Deparvu)

Other Changes
---------------------
* SOLR-16351: Upgrade Carrot2 to 4.4.3, upgrade randomizedtesting to 2.8.0. (Dawid
Weiss)

* SOLR-16245: Make DenseVectorField codec agnostic (Elia Porciani via Alessandro

Benedetti)

* SOLR-15897: Remove <jmx/> from all unit test solrconfig.xml files. (Eric Pugh)
* SOLR-15776: Admin UI is now aware of logged-in user's permissions and can adapt
accordingly (janhoy)

* SOLR-15923: Cue users in 404 message that Solr is running and the url is
incorrect. (Eric Pugh, Mike Drob, Shawn Heisey)

* SOLR-9529: Consistency in how dynamic fields are defined in sample schema.xml

files (Eric Pugh, Trey Grainger)

* SOLR-15886: Remove deprecated showItems configuration value from solrconfig.xml

files (Andy Lester via Eric Pugh)

* SOLR-16114: SolrZooKeeper has been removed in favor of using ZooKeeper directly.

(Mike Drob)

* SOLR-16154: Event listeners submit through core container executor service

instead of separate thread (Mike Drob, Kevin Risden)

* SOLR-16190: Http2SolrClient should make sure to shutdown the executor (Kevin

Risden)

* SOLR-16191: Validate that installed ps utility supports -p flag, so that we do

not inadvertantly stop the wrong process. (Mike Drob, Michael Gibney)

* SOLR-16209: Rolling restart will no longer trigger as much PKI Plugin error
logging. (Mike Drob, Tomás Fernández Löbbe)

* SOLR-16213: Upgrade Jackson to 2.13.3 (janhoy)

* SOLR-16231: Fix ref-guide links across the codebase (Houston Putman)

* SOLR-16304: No more @Slow annotation for tests! (Mike Drob)

* SOLR-16331: Migrate remaining managed-schema files to be managed-schema.xml.

(Eric Pugh)

* SOLR-16332: Upgrade Jetty to 9.4.48.v20220622 (Chris Sabelstrom, janhoy)

* SOLR-16339: Refined "no servers hosting shard" SolrException wording. (Christine

Poerschke)

* SOLR-15342: SolrJ: Split ZooKeeper based dependencies off to a solr-solrj-

zookeeper module /
dependency. (David Smiley, Haythem Khiri)

* SOLR-16324: Upgrade commons-configuration2 to 2.8.0 and commons-text to 1.8

(Kevin Risden)

* SOLR-16296: XmlConfigFile and QueryElevationComponent now use SafeXMLParsing.

(Haythem Khiri)

* SOLR-16369: Avoid XPath in parsing elevate.xml (Haythem Khiri)

* SOLR-16402: Bump google-cloud-bom to 0.178.0 (Kevin Risden)

* SOLR-16141: Upgrade Apache Tika to 1.28.4 (janhoy)

* SOLR-16424: Upgrade Netty to 4.1.82.Final (Kevin Risden)

* SOLR-16421: Upgrade Apache Calcite to 1.32.0 and Avatica to 1.22.0 (Kevin Risden)

* SOLR-16422: Upgrade Apache Zookeeper to 3.8.0 (Kevin Risden)

* SOLR-16423: Upgrade Apache Hadoop to 3.3.4 (Kevin Risden)

* SOLR-16450: Proper handling on watcher registration failure in

ZkStateReader#waitForState() (Hitesh Khamesra via Ishan Chattopadhyaya)

* SOLR-16451: Don't fetch the PRS states while registering the collection watch
(Hitesh Khamesra via Ishan Chattopadhyaya)

* SOLR-16452: Do not update PRS states if local version is newer (Hitesh Khamesra
via noble)

* SOLR-16464: Upgrade commons-text to 1.10.0 (Kevin Risden)

* SOLR-16482: Add an internal envVar for passing options to bin/solr that will not
be overriden by the user. (Houston Putman)

* SOLR-16486: Pin the OS Variant for the default base image of the Dockerfile (to
Ubuntu 20). (Houston Putman, janhoy, Eva Müller)

Build
---------------------
* SOLR-16204: Change Lucene dependency to Lucene 9.1.0 (Elia Porciani via
Alessandro Benedetti)

* SOLR-16303: Change Lucene dependency to Lucene 9.3.0 (Mike Drob)

* SOLR-16053: Upgrade scriptDepVersions (Kevin Risden)

* SOLR-16050: Upgrade to errorprone 2.11.0 (Kevin Risden)

* Upgrade forbiddenapis to 3.3 (Uwe Schindler)

* SOLR-16258: Gradle no longer uses "errorprone" by default (Dawid Weiss, Uwe

Schindler, hossman)

* SOLR-16264: Set versions for all Antora build tools (Houston Putman)

* SOLR-16299: Ref-guide examples are no longer symlinked from SolrJ, they live in
the ref-guide module (Houston Putman)

* SOLR-16298: Clean the javadocs directory before building (Houston Putman, Mike
Drob, Uwe Schindler)

================== 9.0.0 ==================

New Features
---------------------
* SOLR-14440: Introduce new Certificate Authentication Plugin to load Principal
from certificate subject. (Mike Drob)

* SOLR-13528 Rate Limiting in Solr (Atri Sharma, Mike Drob)

* SOLR-14749: Provide a clean API for cluster-level event processing.

Improve support for arbitrary container-level plugins. Add ClusterSingleton
support for plugins that require only one active instance in the cluster. (ab,
noble)

* SOLR-14613, SOLR-15019: Autoscaling replacement using placement plugins (ilan,

ab, noble)

* SOLR-15055: Re-implement 'withCollection'. This also adds the placement plugin

support
for rejecting replica / collection deletions that would violate placement
constraints. (ab, ilan)

* SOLR-15130: Support for per-collection replica placement node sets, a.k.a "node
type"
placements. (ab, ilan)

* SOLR-8138: Simple UI for issuing SQL queries (Michael Suzuki via Eric Pugh)

* SOLR-14787: Payload check query parser now supports inequalities. (Kevin Watters,
Gus Heck)

* SOLR-15164: Implement Task Management Interface

(Atri Sharma, with extensive review and perf testing by Anshum Gupta, Mike Drob
and Houston Putman)

* SOLR-15300: Report collection and shard "health" state in CLUSTERSTATUS response.

(ab, janhoy)

* SOLR-15423: JWTAuthPlugin now supports separate config for what SSL certs to
trust when talking to IdPs (janhoy)

* SOLR-15694: Node roles framework, allowing restriction of certain nodes to

certain tasks (Ishan Chattopadhyaya, noble)

* SOLR-15197: Support temporal graph queries with DAY and WEEKDAY windows (Joel
Bernstein)

* SOLR-15914: Official Solr modules (contribs) can now easily be added to shared
class path by environment variable
'SOLR_MODULES' or system property 'solr.modules'. E.g:
SOLR_MODULES=extracting,langid
(janhoy, David Smiley, Houston Putman)

* SOLR-15880: Introduce support for k nearest neighbors search (Alessandro

Benedetti, Elia Porciani)

* SOLR-14660: Move HDFS support to a new HDFS module (Istvan Farkas, Kevin Risden)

* SOLR-13989: Move Hadoop Authentication support to a new hadoop-auth module (Kevin

Risden)

* SOLR-15904: Move SQL support to new sql module (Kevin Risden)

Improvements
----------------------
* LUCENE-8984: MoreLikeThis MLT is biased for uncommon fields (Andy Hind via Anshum
Gupta)

* SOLR-14223: PKI Auth can bootstrap from existing key files instead of creating
new keys on startup (Mike Drob)
* SOLR-15153: Admin UI: Collection selector drop down does not sort collections
(Edward Ribeiro)

* SOLR-11725: Use corrected sample formula for computing stdDev and variance in
JSON aggregations
(hossman, Munendra S N, yonik)

* SOLR-14387: SolrClient.getById() will escape comma separator within ids (Markus

Schuch via Mike Drob)

* SOLR-15327: Fix typos in the code base (Edward Ribeiro via Eric Pugh)

* SOLR-10814: Add short-name feature to RuleBasedAuthz plugin (Mike Drob,

Hrishikesh Gadre)

* SOLR-7683: Introduce support to identify Solr internal request types (Atri

Sharma, Hrishikesh Gadre)

* SOLR-14799: JWT authentication plugin only requires "sub" claim when

principalClaim=sub. (Erik Hatcher)

* SOLR-14878: Report solr.xml's coreRootDirectory property via System Settings API,

when set (Alexandre Rafalovitch)

* SOLR-14880: Support coreRootDirectory setting when creating new cores from

command line in standalone mode
(Alexandre Rafalovitch)

* SOLR-14926, SOLR-14926, SOLR-13506: Modernize and clean up search results

clustering module. This issue upgrades
the clustering module to the new Carrot2 4.x line, dropping several CVE-prone
dependencies along the way.
The parameters and configuration of the module extensions have changed. The
documentation in Solr ref guide
has been rewritten from scratch to be up to date. Clustering code has been
rewritten from scratch to work
properly regardless of the mode (standalone, distributed). The API has been
stripped of ancient, unused, interfaces
and simplified. (Dawid Weiss)

* SOLR-14972: Prometheus: Change default port of prometheus exporter to 8989

because it clashed with default
embedded zookeeper port (janhoy)

* SOLR-15011: /admin/logging handler will now propagate setLevel (log threshold) to

all nodes
when told to. The admin UI now tells it to. (Nazerke Seidan, David Smiley)

* SOLR-15100: Make the ConfigSetService pluggable/configurable via <string

name="configSetService" /> in solr.xml
(baisui)

* SOLR-15185: Various optimizations to the {!hash} QParser, typically used by the

parallel()
streaming expression. The hash algorithm changed. (David Smiley)

* SOLR-2852: SolrJ: remove Woodstox dependency. It was never truly required there.
Software doing lots of XML processing can choose to add it or alternatives if
they wish.
 (David Smiley)

* SOLR-15161: Don't encourage users to hack JSON response mimetype by documenting

in examples how to
specify wt=json use mimetype of text/plain. (Eric Pugh)

* SOLR-15276: V2 API call to look up async request status restful style of

"/cluster/command-status/1000" instead
of "/cluster/command-status?requestid=1000". (Eric Pugh)

* SOLR-15274: The QueryElevationComponent now supports loading elevation file

changes on commits.
This doesn't work in SolrCloud (but may someday). QEC no longer supports a
config file in
the data dir. (David Smiley)

* SOLR-14185: Added DocSet.iterator(LeafReaderContext) and some related changes

that may add
a minor performance boost to some cases (e.g. interval facets). Reduced need for
Filter.java.
(Michael Gibney, David Smiley)

* SOLR-15329: Improve HDFS Directory size calculation (Andras Salamon via Mike
Drob)

* SOLR-15340: Rename shardsWhitelist and extract AllowListUrlChecker to use it more

broadly. (Bruno Roustant)

* SOLR-15414: Use ConfigSet API instead of Zookeeper data node to list out
configsets available in Solr Admin UI.
(Nazerke Seidan via Eric Pugh)

* SOLR-15421: ConfigSet API also checks for solrconfig.xml when checking the
existence of a configset
(Andras Salamon via David Smiley)

* SOLR-15392: Distributed Tracing request span operation names are now composed of
a command/verb
and a templated path. The collection or core name is now in the db.instance tag.
(David Smiley)

* SOLR-15453: Update the content security policy in Jetty to allow image requests
from local host and prevent security
errors on the client side. (MarcusSorealheis via Houston Putman)

* SOLR-15362: Let core and collection dropdowns in Admin UI float wide to see
entire core or collection name.
(Matthias Krepp, Eric Pugh)

* SOLR-15044: When indexing nested docs via JSON, it is no longer necessary to

provide child doc IDs.
This was already working for XML & "javabin"/SolrJ. Previously, omitting the ID
would be confused
for a partial/atomic update. (David Smiley)

* SOLR-10887: Migrate "managed-schema" file naming to "managed-schema.xml" file

name, with a fallback to the legacy
"managed-schema". (Eric Pugh, David Smiley)
* SOLR-15630: Logging MDC values no longer include a hardcoded prefix, allowing
custom logging configurations access to
the plain values. The default log4j2.xml PatternLayout has been updated to ensure
the values are formatted with the
existing prefixes. (hossman)

* SOLR-15650: Choosing lucene defType in Solr Admin now is passed explicitly

through UI, not relying on default
solrconfig.xml behavior. (Eric Pugh)

* SOLR-12848: SolrJ and the server can now recognize some standard Java system
properties like
for an HTTP proxy. This is only for the Apache HttpClient based SolrJ
communication, not Jetty.
(Shawn Heisey, David Smiley)

* SOLR-15301: Eliminate repetitive index size calculation for Solr metrics (Andras
Salamon)

* SOLR-15617: Add kerberos.name.rules.mechanism support to KerberosPlugin (Kevin

Risden)

* SOLR-15278: Add V2 equivalent to allow deleting async collection list of

statuses. (Eric Pugh)

* SOLR-11623: Every request handler in Solr now implements PermissionNameProvider

to explicitly decide on what security
permissions are required to access the handler (janhoy, Hrishikesh Gadre, David
Smiley)

* SOLR-15705: A delete-by-id command is forwarded to all shards when using the

CompositeId router with a router field
and the route is missing from the command. (Michael Kosten via Christine
Poerschke, David Smiley, Eric Pugh)

* SOLR-15790: SearchHandler now includes the `rid` value in the Logging MDC for the
duration of the request, allowing
custom logging configurations to include it. (hossman)

* SOLR-15785: Custom node request handlers/endpoints that weren't in packages can

now be located in JARs in solr-home/lib,
not just WEB-INF/lib. (Nazerke Seidan, David Smiley)

* SOLR-15376: Accept "Long" values for

CollectionAdminRequest.CreateTimeRoutedAlias.setMaxFutureMs
to support durations greater than ~ 25 days. (Nahian-Al Hasan, Gus Heck,
Christine Poerschke)

* SOLR-15427: Nested docs: [child limit=...] now defaults to -1 which is

interpreted as unlimited. (David Smiley)

* SOLR-15786: Add the "films" example to SolrCLI via -e films parameter. (Eric
Pugh)

* SOLR-15834: Films example readme needs updating, including useParams support for
multiple algorithms. (Eric Pugh)

* SOLR-15824: Improved Query Screen handling of raw query parameters. (Betul Ince
via Tim Potter, Eric Pugh)
* SOLR-15803: Compute concurrent replica assignment requests together, using the
shared context to better distribute
replicas. (Houston Putman)

* SOLR-15213: Atomic updates: "add" now uses add-or-replace logic for child
documents. They can
also themselves be atomic updates. (James Ashbourne, Endika Posadas via David
Smiley)

* SOLR-10321: Highlighting (hl.method=unified): When there are no highlights for a

field, don't
return the field in the response at all. (David Smiley)

* SOLR-15890: Add a limit to the Admin SQL panel if one is not included in the stmt
(Joel Bernstein)

* SOLR-15887: Remove <jmx/> from shipped solrconfig.xml. (Eric Pugh)

* SOLR-14608: Faster sorting for the /export handler (Joel Bernstein)

* SOLR-15209: The old LegacyAssignStrategy has been refactored into the

SimplePlacementPlugin. This is still the default
placement policy for Solr. (Houston Putman, Ilan Ginzburg)

* SOLR-15257: Replace DocSet.getTopFilter with DocSet.makeQuery. Create DocSetQuery

which is a Query
and DocSetProducer. (Collins Abanda, Mike Drob, Tim Potter, David Smiley, Michael
Gibney)

* SOLR-12336: Remove Filter, SolrFilter and SolrConstantScoreQuery. Filter no

longer has a need to exist due to multiple JIRA issues,
implemented by a number of issues. With Filter going away, there is no longer
need for SolrConstantScoreQuery. (Collins Abanda, Mike Drob, Tim Potter, David
Smiley, Michael Gibney)

* SOLR-14916: Add split parameter to timeseries Streaming Expression (Joel

Bernstein)

* SOLR-14686: Logs: Removed the "[corename]" prefix of some SolrCore logs that has
become redundant
with MDC. (David Smiley)

* SOLR-15884: Backup responses now use a map to return information instead of a

list (Houston Putman, Christine Poerschke)

* SOLR-9376: Raw value DocTransformers (`[xml]`, `[json]`, `[geo w=GeoJSON]`) now

work
in a distributed/SolrCloud context (Michael Gibney)

* SOLR-15755: Parse collection, shard, core, replica info from newSearcher events
in postlogs tool (Joel Bernstein)

* SOLR-9575: Solr no longer requires a solr.xml to be present in $SOLR_HOME. If one

is not found, Solr will load
the one in $SOLR_TIP/server/solr/solr.xml. Solr also does not require a zoo.cfg
in $SOLR_HOME if started with
embedded zookeeper. (janhoy)
* SOLR-11905: New Admin UI Query screen input boxes for JSON query and facet DSL
(janhoy)

* SOLR-15556: Migrate the Ref Guide to be built with Antora, enabling many new
features
(Cassandra Targett, Houston Putman, Mike Drob)

* SOLR-15965: Use better signatures for the PKI Authentication plugin. (Mike Drob)

* SOLR-15982: Add end time value to backup response, standardize backup response
key names and date formats (Artem Abeleshev, Christine Poerschke, Houston Putman)

* SOLR-14401: Metrics: Only SearchHandler and subclasses have "local" metrics now.
It's now tracked as if it's another handler with a "[shard]" suffix, e.g.
"/select[shard]".
There are no longer ".distrib." named metrics; all metrics are assumed to be such
except
"[shard]". The default Prometheus exporter config splits that component to a new
label
named "internal". The sample Grafana dashboard now filters to include or exclude
this.
(David Smiley)

* SOLR-16088: De-couple Http2SolrClient and ContentStreamBase from org.apache.http

(janhoy)

* SOLR-15286: A brand new follower in the legacy mode should wait to replicate
index before reporting
healthy (Pushkar Raste, Anshum Gupta)

* SOLR-15590: Start CoreContainer via ServletContextListener (Gus Heck)

* SOLR-16131: Jetty threads' contextClassLoaders are set to the Solr core's

resourceLoader's classLoader.
This allows for custom plugins and Solr modules to rely on 3rd part dependencies
that use the thread contextClassLoader.
(Houston Putman)

Build
---------------------
* LUCENE-9077 LUCENE-9433: Support Gradle build, remove Ant support from trunk
(Dawid Weiss, Erick Erickson, Uwe Schindler et.al.)

* LUCENE-9411: Fail compilation on warnings, 9x gradle-only (Erick Erickson, Dawid

Weiss)
Deserves mention here as well as Lucene CHANGES.txt since it affects both.

* SOLR-13671: Remove check for bare "var" declarations in validate-source-patterns

(Erick Erickson, Alex Bulygin, janhoy)

* SOLR-14857: Add optional gradle property to run docker tests in parallel

(hossman, Houston Putman)

* SOLR-15793: Pin http_parser.rb to specific version to allow Solr Ref Guide on

Jekyll to run (Eric Pugh)

* SOLR-15867: Make the gradle build ready for 9.0 release (janhoy, Houston Putman,
Dawid Weiss)
* SOLR-15852: Update dev-tools/scripts for the 9.0 release, including releaseWizard
(janhoy)

* SOLR-15891: The Solr Docker image uses the default file permissions from the
tarball artifact (Houston Putman, janhoy)

* SOLR-15603: Add an option to activate Gradle build cache, build task cleanups
(Alexis Tual, Dawid Weiss)

* SOLR-15613: Enforce error-prone checks to catch all URLEqualsHashCode violations

and use java.net.URI instead
(Collins Abanda)

* SOLR-15670: Introduce Gradle parameter to skip ref-guide tasks (Houston Putman,

Dawid Weiss)

* SOLR-15984: Ensure all used dependencies are declared (Kevin Risden)

* SOLR-15987: Upgrade slf4j to 1.7.35 and remove ant-compat/force-versions.gradle

(Kevin Risden)

* SOLR-15992: Globally forbid and exclude known bad dependencies (Kevin Risden)

* SOLR-12228: Move bin-test scripts to packaging project and create gradle task for
running them. (Mike Drob)

Docker
----------------------
* SOLR-14789: Migrate docker image creation from docker-solr repo to solr/docker.
(Houston Putman, Martijn Koster, Tim Potter, David Smiley, janhoy, Mike Drob)

* SOLR-14790: Docker: Move Solr Docker image documentation to the ref guide.
(Houston Putman)

* SOLR-15322: Solr releases now contain everything needed to build runnable docker
images (Houston Putman, hossman)

* SOLR-15949: Docker: the official image now uses Java 17 provided by Eclipse
Temurin. Formerly it was Java 11 from OpenJDK.
(janhoy, David Smiley)

* SOLR-15891: The Solr Docker image uses the default file permissions from the
tarball artifact (Houston Putman, janhoy)

* SOLR-14001: Docker: Removed /var/solr initialization from the Dockerfile; depend

on init_var_solr.sh instead.
This leads to more consistent behavior no matter how /var/solr is mounted.
* init_var_solr.sh is now invoked by docker-entrypoint.sh; not in a bunch of
other places.
* as before, you can set NO_INIT_VAR_SOLR=1 to short-circuit this.
* init_var_solr.sh no longer echo's anything. For verbosity, set VERBOSE=yes.
(David Smiley)

* SOLR-14957: Docker, Prometheus: Add Prometheus Exporter to docker PATH. Fix

classpath issues.
(Houston Putman)

* SOLR-14949: Docker: Ability to customize the FROM image when building.

(Houston Putman)
* SOLR-15335: Docker: Solr has capability to build functionally-identical local and
official Docker image.
(hossman, Houston Putman)

Deprecation Removals
----------------------
* SOLR-15203: JWT Auth plugin: Remove deprecated parameter name jwkUrl in favour of
jwksUrl for the JWK Url. (Eric Pugh)

* SOLR-15608: Remove deprecated methods, classes and constructors from solrj

clients (janhoy)

* SOLR-14272: Remove autoReplicaFailoverBadNodeExpiration and

autoReplicaFailoverWorkLoopDelay for 9.0 as it was
deprecated in 7.1 (Anshum Gupta)

* SOLR-14197: SolrResourceLoader: remove deprecated methods and do other

improvements. (David Smiley)

* SOLR-9909: The deprecated SolrjNamedThreadFactory has been removed. Use

SolrNamedThreadFactory instead.
(Andras Salamon, shalin)

* SOLR-14783: Remove Data Import Handler (DIH), previously deprecated (Alexandre

Rafalovitch)

* SOLR-14035: Remove deprecated preferLocalShards=true support in favour of the

shards.preference=replica.location:local
alternative. (Alex Bulygin via Christine Poerschke)

* SOLR-14934: Remove redundant deprecated "solr.solr.home" logic (hossman)

* SOLR-14034: Remove deprecated min_rf references (Tim Dillon)

* SOLR-13893: Remove support to read BlobRepository's max jar size from deprecated
`runtme.lib.size` system property
(Erick Erickson, Kesharee Nandan Vishwakarma, Munendra S N)

* SOLR-12720: Remove support for `autoReplicaFailoverWaitAfterExpiration`.

(marcussorealheis, shalin)

* SOLR-12823: Remove /clusterstate.json support, including support for collections

created with stateFormat=1,
as well as support for Collection API MIGRATESTATEFORMAT action and support for
the legacyCloud flag (Ilan Ginzburg).

* SOLR-12847: Remove support for maxShardsPerNode. (ab)

* SOLR-14244: Remove ReplicaInfo. (ab)

* SOLR-14654: Remove plugin loading from .system collection (noble)

* SOLR-14944: Remove the "spins" metrics - support for detection of spinning disks
has been
removed in LUCENE-9576. (ab)

* SOLR-15341: Remove indexHeapUsageBytes info from /admin/segments and /admin/luke

because it's no
 longer available in Lucene -- LUCENE-9387. (janhoy, David Smiley)

* SOLR-15416: Remove metrics history collection (and MetricsHistoryHandler). (ab)

* SOLR-13138: Remove deprecated LegacyBM25SimilarityFactory class. (Christine

Poerschke, janhoy)

* SOLR-15716: Remove deprecated SolrException.ignorePatterns and related code

(hossman)

* SOLR-12336: Remove Filter, SolrFilter and SolrConstantScoreQuery. Filter no

longer has a need to exist due to multiple
JIRA issues, implemented by a number of issues. With Filter going away, there is
no longer need for
SolrConstantScoreQuery. (Collins Abanda, Mike Drob, Tim Potter, David Smiley,
Michael Gibney)

Other Changes
----------------------
* SOLR-14656: Autoscaling framework removed (Ishan Chattopadhyaya, noble, Ilan
Ginzburg)

* SOLR-14616: CDCR support removed (Ishan Chattopadhyaya)

* LUCENE-9391: Upgrade HPPC to 0.8.2. (Haoyu Zhai)

* SOLR-10288: Remove non-minified JavaScript from the webapp. (Erik Hatcher,

marcussorealheis)

* SOLR-13655: Upgrade Collections.unModifiableSet to Set.of and Set.copyOf (Atri

Sharma via Tomás Fernández Löbbe)

* SOLR-13797: SolrResourceLoader no longer caches bad results when asked for wrong
type (Mike Drob)

* LUCENE-9092: Upgrade Carrot2 to 3.16.2 (Dawid Weiss).

* LUCENE-9080: Upgrade ICU4j to 62.2 and make regenerate work (Erick Erickson)

* SOLR-14271: Remove duplicate async id check meant for pre Solr 8 versions (Anshum
Gupta)

* SOLR-14272: Remove autoReplicaFailoverBadNodeExpiration and

autoReplicaFailoverWorkLoopDelay for 9.0 as it was
deprecated in 7.1 (Anshum Gupta)

* SOLR-14258: DocList no longer extends DocSet. (David Smiley)

* SOLR-14256: Remove HashDocSet; add DocSet.getBits() instead. DocSet is now

strictly immutable and ascending order.
It's now locked-down to external extension; only 2 impls exist. (David Smiley)

* SOLR-14012: Return long value for unique and hll aggregations irrespective of
shard count (Munendra S N, hossman)

* SOLR-14322: AbstractFullDistribZkTestBase.waitForRecoveriesToFinish now takes a

timeout and time unit instead of
assuming that we are passed value in seconds. (Mike Drob)
* SOLR-14420: AuthenticationPlugin.authenticate accepts HttpServletRequest instead
of ServletRequest. (Mike Drob)

* SOLR-14429: Convert .txt files to properly formatted .md files. (Tomoko Uchida,
Uwe Schindler)

* SOLR-14412: Automatically set urlScheme to https when running secure solr with
embedded zookeeper. (Mike Drob)
Do not erroneously set solr.jetty.https.port system property when running in http
mode (Upendra Penegalapati)

* SOLR-14014: Introducing a system property that allows users to disable the Admin
UI, which is enabled by default.
If you have security concerns or other reasons to disable the Admin UI, you can
modify `SOLR_ADMIN_UI_DISABLED`
`solr.in.sh`/`solr.in.cmd` at start. (marcussorealheis)

* SOLR-14702: All references to "master" and "slave" replaced with "leader" and
"follower" (MarcusSorealheis,
Erick Erickson, Tomás Fernández Löbbe)

* SOLR-14846: Backup/Restore classes no longer take Optional method parameters and

will accept nulls instead. (Mike Drob)

* LUCENE-9531: Consolidated CharStream and FastCharStream classes: these have been

moved from each query parser
package to org.apache.lucene.queryparser.charstream (Dawid Weiss).

* SOLR-9607: Remove /terms configuration from solrconfig.xml, as implicit

definition is up-to-par now (Alexandre Rafalovitch)

* SOLR-14036: Remove distrib=false from /terms handler's default parameters (David

Smiley, Munendra S N)

* SOLR-14829: Cleanup the documentation for Request Handlers and Search Components
(Alexandre Rafalovitch)

* SOLR-10370: ReplicationHandler should fetch index at fixed delay instead of fixed

rate (Cao Manh Dat)

* SOLR-14930: Removed rule based replica placement (noble)

* SOLR-14912: Clean up solr-extraction module to produce solr-extraction-* jar

(instead of solr-cell-*). (Dawid Weiss)

* SOLR-14915: Prometheus: Reduced dependencies from Solr server down to just SolrJ.
Don't add WEB-INF/lib.
* Can run via gradle, "gradlew run"
* Has own log4j2.xml now
* Was missing some dependencies in lib/; now has all except SolrJ & logging.
(David Smiley, Houston Putman)

* SOLR-15016: Replica placement plugins should use container plugins API / configs.
(ab, ilan)

* SOLR-14297: Replace commons-codec Base64 with JDK11 Base64 (Andras Salamon via
Houston Putman)

* SOLR-15113: Do not attempt to start Solr server when embedded ZK fails (Mike
Drob)

* SOLR-14067: StatelessScriptUpdateProcessorFactory moved to it's own

/module/scripting/ package instead of shipping as
part of Solr due to security concerns. Renamed to ScriptUpdateProcessorFactory
for simpler name. (Eric Pugh)

* SOLR-15118: Switch /v2/collections APIs over to the now-preferred annotated-POJO

implementation approach (Jason Gerlowski)

* SOLR-14928: Allow cluster state updates to be done in a distributed fashion

without going through Overseer (Ilan Ginzburg)

* SOLR-15002: Upgrade httpclient to 4.5.13 and httpcore to 4.4.13 (Andras Salamon

via janhoy)

* SOLR-15121: Move XSLT (tr param) response writer and update request handler to
scripting module. (Eric Pugh, David Smiley)

* SOLR-15169: SolrPaths.assertPathAllowed normalization problem (Andras Salamon via

janhoy)

* SOLR-15292: SignatureUpdateProcessorFactory will fail to initialize if it is used

in a SolrCloud cluster in a way that is
known to be problematic with multiple replicas. (hossman)

* SOLR-15258: Consolidate ConfigSet handling (C.R.U.D.) into ConfigSetService to

allow for viable
alternative implementations of where ConfigSets come from.
(Nazerke Seidan, David Smiley)

* SOLR-15280: Fixed misleading error message in HdfsDirectoryFactory (Andras

Salamon via janhoy)

* SOLR-15146: Allow Collection API and Config Set API to be done in a distributed
fashion without going through Overseer
(Ilan Ginzburg)

* SOLR-15356: "mergeindexes" should not use UninvertingReader (FieldCache). (David

Smiley)

* SOLR-15356: UninvertDocValuesMergePolicy was removed because it's incompatible

with Lucene 9.
(David Smiley, Christine Poerschke)

* SOLR-15283: Overhaul distributed-tracing. Solr's "samplePercentage" is gone;

tracing is either
enabled and you always have trace IDs or it isn't, based on whether you configure
a tracing plugin.
Reporting of traces to a server like Zipkin is a configuration matter of the
tracing plugin you
use. The Jaeger plugin is now completely configurable via system properties or
environment vars,
not by values in solr.xml. (David Smiley)

* SOLR-14166: Non-cached filter queries are now pushed down to Lucene, possibly
benefiting from
TwoPhaseIterator, which can make a big difference for some queries. (David
Smiley)
* SOLR-15369: PackageStoreAPI will only be loaded in SolrCloud mode (Mike Drob)

* SOLR-15222: userfiles dir will only be created in SolrCloud mode (Mike Drob)

* SOLR-15401: Document the new NorwegianNormalizationFilter introduced in LUCENE-

9929. (janhoy)

* SOLR-15409: Upgrade to Zookeeper 3.7.0 (Mike Drob)

* SOLR-15439: Upgrade jose4j library used for JWT authentication processing, to

v0.7.7 (janhoy)

* SOLR-15385, SOLR-15535: Address many rawtypes warnings, resulting in several

modified signatures in the public API.
(Mike Drob, David Smiley, Christine Poerschke)

* SOLR-15470: The binary distribution no longer contains test-framework jars

(janhoy)

* SOLR-15471: Rename lang id whitelist parameter to allowlist (Mike Drob)

* SOLR-14341: SolrCloud: move the reference a collection has to a configSet in

ZooKeeper from the
collections/collectionName into its state.json. For many-collection clusters,
this is an
optimization when the cluster status is fetched. (Nazerke Seidan, David Smiley)

* SOLR-15517: Remove unnecessary no-op implementation of SolrCoreAware in

ExpandComponent and TermVectorComponent.
(Christine Poerschke)

* SOLR-15309: Add missing IntelliJ IDEA entries to the .gitignore file (Pushkar
Raste via Anshum Gupta)

* SOLR-15428: Integrate the OpenJDK JMH micro benchmark framework for micro
benchmarks and performance comparisons
and investigation. (Mark Miller)

* SOLR-15111: Use JDK11 Base64 instead of own implementation (Andras Salamon via
janhoy)

* SOLR-15612: Remove unnecessary https8 jetty module (Houston Putman)

* SOLR-15606: Rename (deprecated) Insanity.java to NumericHidingLeafReader.java

adjusting callers to match.
(Christine Poerschke)

* SOLR-15480: Make Tuple copy constructor, clone and merge consistent w.r.t.
markers (EOF, EXCEPTION), field names
and labels. (John Durham, Mike Drob, Christine Poerschke)

* SOLR-15692: Fix parsing of -zkHost flag for the bin/solr commands (janhoy)

* SOLR-15699: Removal of the (unused since 5.0)

"updateHandler/indexWriter/closeWaitsForMerges" solrconfig.xml element.
(Christine Poerschke)

* SOLR-15728: Remove dead, unused log rotation code from SolrCLI (janhoy)
* SOLR-15791: Remove remaining unused <admin/> clauses from solrconfigs. (Eric
Pugh)

* SOLR-15784: Remove SolrJ dependency on commons-io. (Mike Drob)

* SOLR-15809: Get rid of blacklist/whitelist terminology

JWTAuthPlugin parameter 'algWhitelist' is now 'algAllowlist'
Environment variables SOLR_IP_WHITELIST and SOLR_IP_BLACKLIST are no longer
supported, but replaced with
SOLR_IP_ALLOWLIST and SOLR_IP_DENYLIST. (janhoy)

* SOLR-15807: New LogListener class for tests to use to make assertions about what
Log messages should or should not
be produced by a test (hossman)

* SOLR-14660: Move HDFS support to a new HDFS module (Istvan Farkas, Kevin Risden)

* SOLR-15907: Move JWT Authentication plugin to module 'jwt-auth' (janhoy)

* SOLR-15845: Add a new SolrVersion class to manage Solr's version independently

from the Lucene version we consume (janhoy)

* SOLR-15912: Upgrade guava to version 31.0.1-jre (janhoy)

* SOLR-14858: Add the server WEB-INF/lib directory to the classpath for the solr-
exporter script. Will allow the script
to work when the dist/solrj-lib jars are missing in the Docker image. (Houston
Putman)

* SOLR-15124: Removed three core level admin API endpoints because they are already
registered at the node level where they
really belong: /admin/threads, /admin/properties, /admin/logging (Nazerke Seidan,
David Smiley)

* SOLR-14142: Jetty's RequestLog is enabled by default. If you don't want these

logs, you can disable
via SOLR_REQUESTLOG_ENABLED=false. (rmuir, janhoy)

* SOLR-15924: Remove lucene-libs from module packaging. The lucene libraries will
be included in lib/ with all
other dependencies. (Houston Putman)

* SOLR-15934: CloseHook is now an interface with default method implementations

instead of an abstract class. (Mike Drob)

* SOLR-15953: Remove unused LTRThreadModule.preClose CloseHook logic. (Christine

Poerschke)

* SOLR-15916: Remove dist/ from the binary release. The solr-core and solrj jars
will be included in the server
WEB-INF/lib directory, and each individual module's jar will be included in its
directory's lib/ folder. (Houston Putman)

* SOLR-15917: "Contrib Modules" have been renamed to "Modules" and have been moved
from "contrib/" to "modules/".
(Houston Putman)

* SOLR-15632: Upgrade to Caffeine Cache 3.0.5 (Mike Drob)

* SOLR-15954: Move the Prometheus Exporter from "solr/modules/prometheus-exporter"
to "solr/prometheus-exporter".
(Houston Putman)

* SOLR-15957: Add port and scraping information to Solr Prometheus startup logging.
(Houston Putman)

* SOLR-15936: Reduce unnecessary startup logging, such as SSL warnings when SSL not
in use (janhoy)

* SOLR-15950: Create SOLR_HOME/filestore lazily on first use. Do not automatically

create SOLR_HOME/userfiles (janhoy)

* SOLR-15777: Forbid useDocValuesAsStored for ICUCollationField (warn for

luceneMatchVersion < 9.0.0). (Michael Gibney)

* SOLR-12901: Highlighting: hl.method=unified is the new default. (David Smiley)

* SOLR-15991: analysis-extras module tests shouldn't rely on log4j dependency

(Kevin Risden)

* SOLR-15064: Atomic/partial updates to nested documents now _require_ the `\

_root_` field to
clearly show the document isn't a root document. Solr 8 would fallback on the `\
_route_` param but no longer.
(David Smiley)

* SOLR-15943: Simplify HDFS cache to account for LUCENE-10366 / LUCENE-10376 (Kevin

Risden)

* SOLR-15942: Upgrade Hadoop to 3.3.1 (Kevin Risden)

* SOLR-15990: Use google-cloud-bom for gcs-repository dependencies (Kevin Risden)

* SOLR-15989: Upgrade to Tika 1.28.1 (Kevin Risden)

* SOLR-16042: Fix TestSolrCloudWithKerberosAlt.testBasics failure due to minikdc

locale (Kevin Risden)

* SOLR-16043: Fix HDFS tests - "Command processor" thread leak (Mike Drob, Kevin
Risden)

* SOLR-16074: Use PathUtils.deleteDirectory(path) to try to avoid commons-io IO-751

(Houston Putman, Mike Drob, Kevin Risden)

* SOLR-16039: Upgrade to Hadoop 3.3.2 (Kevin Risden)

* SOLR-15223: SolrJ: The project is migrating away from the Apache HttpClient
dependency in favor of
Jetty's client (supporting HTTP/2), to occur over the 9x releases. We deprecated
HttpSolrClient
and friends in favor of Http2SolrClient and equivalents. For SolrCloud, the
former CloudSolrClient
was renamed CloudLegacySolrClient (deprecated), and instead BaseCloudSolrClient
was renamed to
CloudSolrClient and given a Builder for the Jetty based HTTP/2 client. (janhoy,
David Smiley)
* SOLR-16061: CloudSolrClient refactoring: Removed ZK specific methods (Haythem
Khiri, David Smiley, janhoy)

* SOLR-16101: Add empty constructor for SolrInputDocument (Anshum Gupta)

* SOLR-15780: The v2 API is now treated as "experimental". It may change in

backwards-incompatible ways in subsequent releases
as it expands to cover additional functionality and prepares to supplant the v1
API. (Jason Gerlowski)

Bug Fixes
---------------------
* SOLR-15849: Fix the connection reset problem caused by the incorrect use of 4LW
with \n when monitoring zooKeeper status
(Fa Ming).

* SOLR-14546: Fix for a relatively hard to hit issue in OverseerTaskProcessor that

could lead to out of order execution
of Collection API tasks competing for a lock (Ilan Ginzburg).

* SOLR-15162: Allow readOnly parameter to be used with v2 modify collection command

(Eric Pugh)

* SOLR-15334: Return error response when failing auth in PKIAuthPlugin (Mike Drob)

* SOLR-15449: Edismax sow incorrectly affect mm parameter in multi field search

(Alessandro Benedetti, Michael Gibney, David Smiley)

* SOLR-15410: Always use -Xverbosegclog for OpenJ9. (Colvin Cowie via Eric Pugh)

* SOLR-15653: Fix collection creation race that assumes a local clusterstate when
the collection has only just been
created. (Mark Miller)

* SOLR-10529: Solr UI Health Check enable/disable ping Button doesn't work (Oscar
Wang via janhoy)

* SOLR-15783: Prevent Logging MDC values from leaking between request threads, and
set 'trace_id' in MDC as soon as it's
available (hossman)

* SOLR-14781: Removed many unused classes that accumulated over the years. (Bence
Szabo, David Smiley)

* SOLR-15871: Update Log4J to 2.17.1 (weidongkl via janhoy)

* SOLR-15854: Let RealtimeGet component support negative filters (Tomás Fernández

Löbbe)

* SOLR-15501: GCSBackupRepository no longer strictly requires a pointer to a

service account JSON file, allowing users
running within GCP to take advantage of it's "Workload Identity" and other role-
based access feature.
(Jacek Kikiewicz, Martin Stocker via Jason Gerlowski)

* SOLR-15919: Replace File with Path for many ZK operations (Mike Drob)

* SOLR-15259: hl.fragAlignRatio now defaults to 0.33 to be faster and maybe looks

nicer. (David Smiley)
* SOLR-15944: The Tagger's JSON response format now always uses an object/map to
represent each tag instead of an
array, which didn't make sense. (David Smiley)

* SOLR-15842: Async response for backups now correctly aggregates and returns
information
(Houston Putman, Artem Abeleshev, Christine Poerschke)

* SOLR-14595: Consistent overrequest across different facet methods for

`sort:index` JSON Facet field (Michael Gibney, hossman)

* SOLR-15961: Fix bug in PKIAuthenticationPlugin that can cause a request to fail

with 401 Unauthorized instead
of re-fetching expired remote keys from other nodes. (Tomás Fernández Löbbe)

* SOLR-15968: Hide annoying WARN log from bin/solr zk command (janhoy, Mike Drob)

* SOLR-16075: ShowFileHandler path parameter is now validated to be relative to

instance conf dir in standalone mode (janhoy)

* SOLR-15558: Don't wait for zombie processes to exit when stopping. (Colvin Cowie)

* SOLR-16019: UTF-8 parsing errors for parameters should cause a HTTP 400 status
code, not 500 (janhoy, Matthias Pigulla)

* SOLR-15983: Fix ClassCastException in UpdateLog$LogReplayer.doReplay. (Christine

Poerschke, David Smiley)

* SOLR-16090: Better error message when JWT auth SIGNATURE_INVALID during token
parsing (janhoy)

* SOLR-15333: Reduced spurious warn logging by AbstractSpatialPrefixTreeFieldType

field properties (Steffen Moldenhauer, David Smiley, Mike Drob)

* SOLR-15405: Make HDFS handle little endian LUCENE-9047 (Kevin Risden)

* SOLR-16009: Force Calcite's Rel simplify config flag to false to avoid erasing
filters that are meaningful to Solr,
but look like nonsense to Calcite, such as AND'd filters on the same multi-valued
field (Timothy Potter, Kiran Chitturi)

* SOLR-16022: Enforce special character requirements on passwords with length less

than 15 (Timothy Potter)

* SOLR-16143: SolrConfig can miss updates from ZooKeeper when deleting and
recreating file items (Mike Drob)

* SOLR-16164: ConfigSet API returns error if untrusted user creates from _default
configset (Eric Pugh, Kevin Risden)

================== 8.11.2 ==================

Bug Fixes
---------------------
* SOLR-15871: Update Log4J to 2.17.1 (weidongkl via janhoy)

* SOLR-15854: Let RealtimeGet component support negative filters (Tomás Fernández

Löbbe)
* SOLR-15961: Fix bug in PKIAuthenticationPlugin that can cause a request to fail
with 401 Unauthorized instead
of re-fetching expired remote keys from other nodes. (Tomás Fernández Löbbe)

* SOLR-15558: Don't wait for zombie processes to exit when stopping. (Colvin Cowie)

* SOLR-15587: Don't use the UrlScheme singleton on the client-side to determine

cluster's scheme,
only get from ClusterState provider impl (Timothy Potter)

* SOLR-14569: Configuring a shardHandlerFactory on the /select requestHandler

results in HTTP 401 when searching on alias
in secured Solr. (Isabelle Giguere, Jason Gerlowski, Anshum Gupta, Mark Mark
Miller)

* SOLR-15974: Remove Calcite's ENUMERABLE_AGGREGATE_RULE as Solr only supports

push-down for LogicalAggregate (Timothy Potter, Kiran Chitturi)

* SOLR-16009: Force Calcite's Rel simplify config flag to false to avoid erasing
filters that are meaningful to Solr,
but look like nonsense to Calcite, such as AND'd filters on the same multi-valued
field (Timothy Potter, Kiran Chitturi)

* SOLR-16095: Upgrade JQuery to 3.5.1 in velocity contrib module (Bence Szabo via
Jan Høydahl)

* SOLR-16022: Enforce special character requirements on passwords with length less

than 15 (Timothy Potter)

* SOLR-15849: Fix the connection reset problem caused by the incorrect use of 4LW
with \n when monitoring zooKeeper status (Fa Ming).

* SOLR-15116: Support a HEAD request for managed resources. (Eric Pugh)

* SOLR-16089: Fix empty columns in Cloud Node UI Screen when replica is in down
state. (hossman)

* SOLR-16145: Fix very rare NPE in SolrCloud rejoinOverseerElection. (David Smiley)

* SOLR-16168: Spellcheck NPE if invalid dictionary name is provided (Kevin Risden)

* SOLR-15968: Hide annoying WARN log from bin/solr zk command (janhoy, Mike Drob)

* SOLR-16075: ShowFileHandler path parameter is now validated to be relative to

instance conf dir in standalone mode (janhoy)

* SOLR-16019: UTF-8 parsing errors for parameters should cause a HTTP 400 status
code, not 500 (janhoy, Matthias Pigulla)

* SOLR-16143: SolrConfig can miss updates from ZooKeeper when deleting and
recreating file items (Mike Drob)

* SOLR-16199: Improve query syntax construction for SQL LIKE clause with phrases
and wildcards (Kiran Chitturi, Aroop Ganguly, Amrit Sarkar via Timothy Potter)

* SOLR-16191: Validate that installed ps utility supports -p flag, so that we do

not inadvertantly stop the wrong process. (Mike Drob, Michael Gibney)
* SOLR-16213: Upgrade Jackson to version 2.13.3 (janhoy)

* SOLR-16215: Escape query characters in Solr SQL Array UDF functions (Kiran
Chitturi via Timothy Potter)

* SOLR-15965: Use better signatures for the PKI Authentication plugin. (Mike Drob)

* SOLR-16209: Rolling restart will no longer trigger as much PKI Plugin error
logging. (Mike Drob, Tomás Fernández Löbbe)

* SOLR-16227: Solr SQL query parsing fails on multiple LIKE clauses and multiple
terms
(Kiran Chitturi via David Smiley)

* SOLR-16218: failOnVersionConflicts option not working for in-place updates

(Lamine Idjeraoui, Houston Putman, Anshum Gupta)

================== 8.11.1 ==================

Bug Fixes
---------------------
* SOLR-15635: Don't close hooks twice when SolrRequestInfo is cleared twice; or
/export with classic join
closed fromCore if provided (Mikhail Khludnev, David Smiley)

* SOLR-15795: Fix REPLACENODE to not use source node when choosing a target node
for new replicas (Houston Putman)

* SOLR-15804: Admin UI once again can show files in the Core/Collection -> Files
screen. Fixed regression from 8.11.0 (Karl Stoney, janhoy)

* SOLR-15768: Set default permissions for the zookeeper endpoints so users won't
need to configure custom permissions for these (janhoy)

* SOLR-15774: Avoid weird off-by-one errors with Angular's 'chosen' select box
directive for the security and schema-designer screens in Admin UI (Timothy Potter)

* SOLR-13900: Reset index values on authorization rules after deleting by index

(Timothy Potter)

* SOLR-15794: Switching a PRS collection from true -> false -> true results in
INACTIVE replicas (noble)

* SOLR-15813: Schema designer not handling `update.autoCreateFields` stored as a

string (vs. boolean) in the config overlay (Timothy Potter)

* SOLR-15825: Security UI 'hasPermission' check should check if the user has the
"all" permission if the requested permission is not defined
to match how the backend works (Timothy Potter)

* SOLR-15826: ResourceLoader should better respect allowed paths (janhoy)

* SOLR-15828: AuthTool (in SolrCLI) should include the config-read, collection-

admin-read, core-admin-read, and all permissions in the initial security.json
to avoid warnings in the security UI (Timothy Potter)

* SOLR-15199: Pass thru all command actions not explicitly handled by the bin/solr
script, such as 'api', to the SolrCLI Java app (Timothy Potter)
* SOLR-15833: Enable exists queries on spatial types. (Mike Drob, Houston Putman)

* SOLR-15832: Clean-up after publish action in Schema Designer shouldn't fail

if .system collection doesn't exist (Timothy Potter)

* SOLR-15844: Upgrade Velocity from v2.0 to v2.3 (wcmrnd1, janhoy)

* SOLR-8319: Fix NPE in pivot facets, add non-Analyzed query method in FieldType.
(Houston Putman, Isabelle Giguere)

* SOLR-15843: Update Log4J to 2.16 (Mike Drob, janhoy)

* SOLR-15324: Upgrade jaegertracing to 1.6.0 and libthrift to 0.14.1 (wcmrnd1,

janhoy)

* SOLR-15405: Make HDFS handle little endian LUCENE-9047 (Kevin Risden)

================== 8.11.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

New Features
---------------------

* SOLR-7642: opt-in support to create ZK chroot on startup (Timothy Potter, Shawn

Heisey, Mark Miller, Tomas Eduardo Fernandez Lobbe,
Jan Høydahl, Steve Molloy, Isabelle Giguere, David Eric Pugh, Gus Heck, Christine
Poerschke, Houston Putman)

* SOLR-15708: SolrJ support for ConfigSet uploading (ab, hossman)

* SOLR-12666: Add authn & authz plugins that supports multiple authentication
schemes, such as Bearer and Basic (Timothy Potter, janhoy)

* SOLR-15721: Support editing Basic auth config from the security UI when using the
MultiAuthPlugin (Timothy Potter)

Improvements
---------------------

* SOLR-15707: Add the ability to set the AWS Profile per S3Repository (Houston
Putman)

* SOLR-15700: Make include paths in bin/solr consistent (turnertg, janhoy via Eric
Pugh)

Optimizations
---------------------
(No changes)

Bug Fixes
---------------------
* SOLR-15626: The "config-read" permission has been fixed to properly allow access
to `/solr/admin/configs?action=LIST` (Jon Senchyna via Eric Pugh)

* SOLR-15691: Admin UI raises yellow warning even when only case of hostnames
differ (janhoy)
* SOLR-15702: Fix S3Repository to follow BackupRepository.createDirectory() API
contract (Houston Putman)

* SOLR-15696: Incremental backups no longer fail on collections that had previously

seen a 'SPLITSHARD' operation. (Jason Gerlowski)

* SOLR-15628: The SolrException.log() helper method has been fixed to correctly

passes the Throwable to the Logger w/o stringification (hossman)

* SOLR-15676: Fix PeerSync failure due to RealTimeGetComponent returning

duplicates. (Ramsey Haddad, Christine Poerschke, David Smiley)

* SOLR-15722: Delete Replica does not delete the Per replica state (noble)

* SOLR-15766: MultiAuthPlugin should send non-AJAX anonymous requests to the plugin

that allows anonymous requests (Timothy Potter, Eric Pugh)

* SOLR-15706: Incremental backup 'purges' no longer delete all index files(!)

(Jason Gerlowski)

Build
---------------------

* LUCENE-10104, SOLR-15631: Upgrade forbiddenapis to version 3.2. (Uwe Schindler)

Other Changes
---------------------
* SOLR-15678: Allow only known content types in ShowFileRequestHandler (janhoy, Gus
Heck, Mal Aware)

* SOLR-15697: Add ErrorLogMuter to test-framework, deprecating

SolrException.ignorePatterns (hossman)

* SOLR-15761: Remove JdbcStream from Lang (Joel Bernstein)

======================= 8.10.1 =======================

Other Changes
---------------------
* SOLR-15677: Upgrade the following dependencies with vulnerabilities.
- jetty: 9.4.44.v20210927
- tika: 1.27
- commons-compress: 1.21
- netty: 4.1.68.Final
- fasterxml.jackson: 2.12.3
- errorprone: 2.9.0
- gcp-client: 1.32.1
(Houston Putman)

================== 8.10.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

New Features
---------------------
* SOLR-15472: New shards.preference option for preferring replicas based on their
leader status, i.e.
`shards.preference=replica.leader:false` would prefer non-leader replicas. (wei
wang via Timothy Potter)

* SOLR-15208: Add the countDist aggregation to the stats, facet and timeseries
Streaming Expressions (Joel Bernstein)

* SOLR-15277: Schema Designer UI (Timothy Potter, Kiran Chitturi)

* SOLR-15527: Security screen in Admin UI for managing users, roles, and

permissions (Timothy Potter)

* SOLR-15089: Allow backup/restoration to Amazon's S3 blobstore (Andy Throgmorton,

Houston Putman, Jason Gerlowski, Pierre Salagnac, Mike Drob)

Improvements
---------------------
* SOLR-15460: Implement LIKE, IS NOT NULL, IS NULL, and support wildcard * in
equals string literal for Parallel SQL (Timothy Potter, Houston Putman)

* SOLR-15451: SolrSchema (for Parallel SQL) should use PKI principal for internal
request to /admin/luke to get table metadata (Timothy Potter)

* SOLR-15456: Get field type info from luke for custom fields instead of defaulting
to String in Parallel SQL (Timothy Potter)

* SOLR-15461: Upgrade Apache Calcite to 1.27.0. (Mark Miller, Timothy Potter)

* SOLR-15489: Implement OFFSET & FETCH for LIMIT SQL queries (Timothy Potter)

* SOLR-15475: Implement COUNT and APPROX_COUNT_DISTINCT aggregation functions for

Parallel SQL (Timothy Potter)

* SOLR-15499: StatsStream implement ParallelMetricsRollup to allow for tiered

computation of SQL metrics
over collection aliases backed by many collections, potentially with many shards
in each (Timothy Potter)

* SOLR-15525: Read ZK credentials from a file specified using a system property

instead of exposing plain-text
credentials as system properties (Timothy Potter)

* SOLR-15549: ZkStateReader now supports connecting to 9.0 Solr Clouds (Houston

Putman)

* SOLR-15570: Include fields declared in the schema in table metadata (SQL) even if
they are empty (Timothy Potter)

* SOLR-15564: Improve filtering expressions in /admin/metrics. (ab)

* SOLR-15576: Allow filtering on ISO-8601 formatted timestamp literals in SQL WHERE

clause (Timothy Potter)

* SOLR-9853: Project multi-valued fields in SQL query results (Timothy Potter)

* SOLR-15599: Upgrade AWS SDK from v1 to v2 for S3 Repository (Houston Putman)

* SOLR-15609: Upgrade log4j to 2.14.1 (hossman)

* SOLR-11752: Support "Accept-Encoding" header to enable response gzip compression

(Matthew Sporleder, Houston Putman, Shawn Heisey, Jan Høydahl)
* SOLR-15610: log4j-layout-template-json is now included in solr server, allowing
JSON formatted logs to be configurable. (hossman)

* SOLR-15607: Explicitly indent results in Admin Query UI by default (Olivia Falk

via Eric Pugh)

* SOLR-15555: Identical filter queries will no longer race to record a filterCache

entry. Allows for async cache computation instead of
all queries contending for the same set of locks on the cache. Fixed a bug with
ramBytes usage reporting on filterCache.
Upgraded Caffeine Cache to 2.9.2 (Mike Drob)

* SOLR-15572: Improve the default Prometheus exporter config performance. (ab)

Optimizations
---------------------
* SOLR-15433: Replace transient core cache LRU by Caffeine cache. (Bruno Roustant)

* SOLR-15496: Reuse comparator objects in contrib/ltr rescorer classes. (marcosfpr

via Christine Poerschke)

Bug Fixes
---------------------
* SOLR-15311: Support async parameter in MODIFYCOLLECTION
(Nazerke Seidan, Christine Poerschke, David Smiley)

* SOLR-15531: SignificantTerms streaming function should not fail on empty

collections (Benedikt Arnold via Mike Drob)

* SOLR-15482: Cross-collection join fixed to ignore documents that do not contain

value in "from" field (hossman)

* SOLR-14506: Fix NPE in COLSTATUS for collections using implicit routing. (Austin
Weidler, ab)

* SOLR-15575: Propagate request level basic auth creds from the top-level async
CollectionAdminRequest to internally
used async requests, such as async status checking (Timothy Potter)

* SOLR-14758: Fix NPE in QueryComponent.mergeIds when using timeAllowed and sorting

(Bram Van Dam, Uwe Schindler)

* SOLR-15579: Allow for many values in a SQL IN clause; previously using more than
19 would result in
the IN clause being ignored, now users are only limited by the
`maxBooleanClauses` configured for a collection (Timothy Potter)

* SOLR-8889: Fixed various problems in Solr and SolrJ that could cause deleteById
commands with "_route_" information to processed
by the wrong shard, and/or fail when forwarded to replicas from the shard leader.
(Dan Fox, Yuki Yano, hossman)

* SOLR-13209: Fix NPE when no grouping query specified (Collins Abanda via Mike
Drob)

* SOLR-6156: Fix NullPointerException if group.field grouping is used with rows=0

and timeAllowed. (Modassar Ather, Christine Poerschke)
* SOLR-14457: SolrClient leaks connections on compressed responses if the response
is malformed (Houston Putman)

* SOLR-15595: Partial results from shard queries needlessly discarded for queries
without sort field (Michael Kosten via Mike Drob, Eric Pugh)

* SOLR-15620: Download Config button in Schema Designer screen should not require
user to re-login when already authenticated (Timothy Potter)

Other Changes
---------------------
* SOLR-15566: Clarify ref guide documentation about SQL queries with `SELECT *`
requiring a `LIMIT` clause (Timothy Potter)

* SOLR-15573: bin/solr auth utility should provide role bindings for `security-
read` and `config-edit` by default
to protect the security and schema designer screens in the Admin UI (Timothy
Potter)

* SOLR-15486: During node shutdown pausing of updates and waiting for in-flight
update requests to finish
before closing cores is no longer SolrCloud specific. (Christine Poerschke, David
Smiley)

* SOLR-15599: woodstox-core-asl:4.4.1 (org.codehaus) replaced with woodstox-

core:6.2.4 (com.fasterxml) (Houston Putman)

* SOLR-15621: index.html for Admin UI should send Solr version in the request for
JavaScript files (Timothy Potter)

================== 8.9.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

New Features
---------------------
* SOLR-14330: ExpandComponent now supports an expand.nullGroup=true option
(hossman)

* SOLR-15150: New update.partial.requireInPlace=true option to prevent any partial

document updates that can't be done In-Place (hossman)

* SOLR-15087: SolrCloud backups can now be restored to compatible existing

collections, instead of requiring a new collection
for each restore. (Jason Gerlowski, shalin, Cao Manh Dat)

* SOLR-15132: Add temporal graph query to the nodes Streaming Expression (Joel
Bernstein)

* SOLR-15365: Improved Grafana dashboard for Prometheus Exporter with new Solr
Cluster row (janhoy)

* SOLR-15397: Expose zookeeper status in the Prometheus exporter (janhoy)

* SOLR-15090: A new 'gcs-repository' contrib can be used to store and retrieve

backups from Google Cloud Storage. (Jason Gerlowski, Shalin Mangar, Cao Manh Dat)

* SOLR-12697: In contrib/ltr FieldValueFeature support "stored=false

docValues=true" a.k.a. pure DocValues fields.
(Stanislav Livotov, Erick Erickson, Tobias Kässmann, Tom Gilke, Christine
Poerschke)

Improvements
---------------------
* SOLR-15081: Metrics for a core: add SolrCloud "isLeader" and "replicaState".
(David Smiley)

* SOLR-15085: Prevent EmbeddedSolrServer calling shutdown on a CoreContainer that

was passed to it. (Tim Owen via Mike Drob)

* SOLR-14234: Unhelpful message in RemoteExecutionException. (ab)

* SOLR-13608: Backups are now done incrementally by default. Multiple backups can
be stored at the same location, and each
backup will only upload those files that are new since the last backup. (Jason
Gerlowski, shalin, Cao Manh Dat)

* SOLR-15123: Revamp SolrCLI tool's help descriptions for all commands for
consistency and clarity. (Eric Pugh)

* SOLR-15101: Add "list" and "delete" APIs for managing incremental backups (Jason
Gerlowski, shalin, Cao Manh Dat)

* SOLR-15038: Add elevateOnlyDocsMatchingQuery and

collectElevatedDocsWhenCollapsing parameters to query elevation.
(Dennis Berger, Tobias Kässmann via Bruno Roustant)

* SOLR-15191: Enhance hash method of JSON faceting to support EnumFieldType and

perhaps some other/custom field types
too. (Thomas Wöckinger, David Smiley)

* SOLR-15194: Allow Solr to make outbound non SSL calls to a JWT IDP via -
Dsolr.auth.jwt.allowOutboundHttp=true property. (Eric Pugh)

* SOLR-15154: Let Http2SolrClient pass Basic Auth credentials to all requests

(Tomás Fernández Löbbe)

* SOLR-6152: Pre-populating values into search parameters on the query page of solr
admin (jafurrer, daniel6691)

* SOLR-11233: Add optional JAVA8_GC_LOG_FILE_OPTS for bin/solr. (Pranav Murugappan,

Christine Poerschke)

* SOLR-15155: Let CloudHttp2SolrClient accept an external Http2SolrClient Builder

(Tomás Fernández Löbbe)

* SOLR-15156: [child] doc transformer's childFilter param no longer applies query

syntax escaping.
It was inconsistent with other Solr options. It did this escaping since 8.0 but
not prior. (David Smiley)

* SOLR-15337: Avoid XPath in solrconfig.xml parsing (noble)

Optimizations
---------------------
* SOLR-15079: Block Collapse - Faster collapse code when groups are co-located via
Block Join style nested doc indexing.
 Used by default when field=_root_, or explicitly requested for other fields via
hint=block. (Joel Bernstein, hossman)

* SOLR-15120 Reduce duplicated core creation work (Mike Drob, Mark Miller)

* SOLR-15067 ChildDocTransformerFactory uses perSegFilter cache (Bence Szabo via

David Smiley)

* SOLR-15315: Optimise contrib/ltr MultipleAdditiveTreesModel by changing recursion

to loop in tree traversal.
(Suryakant Pandey, Alessandro Benedetti, Christine Poerschke)

Bug Fixes
---------------------
* SOLR-15078: Fix ExpandComponent behavior when expanding on numeric fields to
differentiate '0' group from null group (hossman)

* SOLR-15149: Better exception handling for LTR model creation errors (Alessandro
Benedetti, Christine Poerschke)

* SOLR-13034: Partial (AKA Atomic) updates could encounter "LazyField" instances in

the document
cache and not know hot to deal with them when writing the updated doc to the
update log.
(David Smiley)

* SOLR-15191: Fix JSON Faceting on EnumFieldType if allBuckets, numBuckets or

missing is set.
(Thomas Wöckinger via David Smiley)

* SOLR-15457: Fix JSON Faceting on EnumFieldType -- returned values should not be

internal ordinals.
(Thomas Wöckinger via David Smiley)

* SOLR-15216: Fix for Invalid Reference to data.followers in Admin UI. (Dean

Pearce)

* SOLR-15273: Fix NullPointerException in StoredFieldsShardResponseProcessor that

happened when a field name alias is
used for the unique key field in searches with distributed result grouping.
(limingnihao via Christine Poerschke)

* SOLR-15384: Zookeeper Status handler /admin/zookeeper/status not queryable from

SolrJ (janhoy)

* SOLR-11921: Move "cursorMark" logic from QueryComponent to SearchHandler so it

can work with things like QueryElevationComponent
that modify the SortSpec in prepare(), as well as possible custom "search"
components other then QueryComponent. (hossman)

* SOLR-15383: Solr Zookeeper status page shows green even when some Zookeepers are
not serving requests (janhoy)

* SOLR-11904: Mark ReplicationHandler's polling thread as a Solr server thread so

the PKI Interceptor is activated to
allow PULL replicas to replicate from security-enabled leaders (Timothy Potter,
Torsten Bøgh Köster)

* SOLR-15418: The V2 API threw errors for GET requests to collection handlers
like /select.
(David Smiley)

* SOLR-15424: Solr replication UI wraps ETA time on top of next line (janhoy)

* SOLR-15399: IndexFetcher should not issue a local commit for PULL replicas when
leader's version is zero (Timothy Potter)

Other Changes
---------------------
* SOLR-15118: Deprecate CollectionAdminRequest.getV2Request(). (Jason Gerlowski)

* SOLR-15292: An ERROR is logged if SignatureUpdateProcessorFactory is used in

SolrCloud cluster in a way that is known to
be problematic with multiple replicas. In 9.0 this situation will prevent
collection initialization. (hossman)

* SOLR-15377: Improve IndexFetcher's exception logging upon replication errors.

(Torsten Bøgh Köster, Christine Poerschke)

* SOLR-15416: Metrics history handler and metrics history collection (together with
its
specialized reporters) have been deprecated and will be removed in 9.0. (ab)

* SOLR-14978: Enable OOM Killer Script in Solr Foreground. Simplify getting heap
dumps on OOM. (Mike Drob, Houston Putman)

* SOLR-15316: Upgrade Jetty to 9.4.41.v20210516 (janhoy, Mike Drob)

================== 8.8.2 ==================

Bug Fixes
---------------------

* SOLR-15249: Properly set ZK ACLs on /security.json (Mike Drob)

* SOLR-15288: Hardening NODEDOWN event in PRS collections (noble)

* SOLR-15233: Set doAs param in ConfigurableInternodeAuthHadoopPlugin (Geza Nagy,

Jason Gerlowski, Mike Drob)

* SOLR-15217: Use shardsWhitelist in ReplicationHandler. (Bruno Roustant)

================== 8.8.1 ==================

Bug Fixes
---------------------

* SOLR-15145: System property to control whether base_url is stored in state.json

to enable back-compat with older SolrJ versions.
(Timothy Potter)

* SOLR-15114: Fix bug that caused WAND optimization to be disabled in cases where
the max score is requested (such as
multi-shard requests in SolrCloud) (Naoto Minami via Tomás Fernández Löbbe)

* SOLR-15136: Reduce excessive logging introduced with Per Replica States feature
(Ishan Chattopadhyaya)
* SOLR-15138: Collection creation for PerReplicaStates does not scale to large
collections as well as regular collections
(Mike Drob, Ilan Ginzburg, noble, Ishan Chattopadhyaya)

* SOLR-15135: Admin UI display collections with perReplicaState=true in graph view

(Timothy Potter)

================== 8.8.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Upgrade Notes
---------------------

* Internal logic for identifying 'Solr Home' has been refactored to make testing
less error prone. Plugin
developers using SolrPaths.locateSolrHome() or 'new SolrResourceLoader' should
check deprecation warnings as existing
some existing functionality will be removed in 9.0. See SOLR-14934 for more
details.

New Features
---------------------

* SOLR-14907: Add v2 API for configSet upload, including single-file insertion.

(Houston Putman)

* SOLR-14560: Add interleaving support in Learning To Rank. (Alessandro Benedetti,

Christine Poerschke)

* SOLR-15052: Reducing overseer bottlenecks using per-replica states (noble, Ishan

Chattopadhyaya)

Improvements
---------------------
* SOLR-14942: Reduce leader election time on node shutdown by removing election
nodes before closing cores.
(Cao Manh Dat, Mike Drob, hossman, shalin)

* SOLR-14955: Add env var options for the Prometheus Exporter bin scripts (Houston
Putman)

* SOLR-14683: Metrics API should ensure consistent placeholders for missing values.
(ab)

* SOLR-14977 : ContainerPlugins should be configurable with custom objects (noble,

ab)

* SOLR-15015 : Add interleaving algorithm parameter support in Learning To Rank

(Alessandro Benedetti)

* SOLR-8673: Make JSON Facets extendable (Tim Owen via Mikhail Khludnev)

* SOLR-14965: metrics: Adds two metrics to the SolrCloud Overseer:

solr_metrics_overseer_stateUpdateQueueSize
and solr_metrics_overseer_collectionWorkQueueSize with corresponding entries in
the the Prometheus exporter's
default/stock configuration. (Saatchi Bhalla, Megan Carey, Andrzej Białecki,
David Smiley)

* SOLR-14827: Refactor schema loading to not use XPath (noble)

* SOLR-14987: Reuse HttpSolrClient per node vs. one per Solr core when using
CloudSolrStream (Timothy Potter)

* SOLR-14413: Allow timeAllowed and cursorMark parameters together. (John Gallagher

via Mike Drob)

* SOLR-15062: /api/cluster/zk/ls should give the stat of the current node (noble)

* SOLR-15069: [child]: the parentFilter parameter is now fully optional and perhaps
obsolete.
(David Smiley)

* SOLR-14923: The use case of atomic/partial updates to child documents no longer

requires that
_root_ be stored or have docValues, and no longer requires the _nest_path_ field.
However it
now requires that the client pass a _root_ field on these updates to point to the
root ID.
(David Smiley)

* SOLR-15059: Add panels to the default Grafana dashboard for query performance
monitoring, includes updates
to the Prometheus exporter to export query performance metrics, such as QPS and
p95. (Timothy Potter)

* SOLR-15040: Improvements to postlogs timestamp handling (Joel Bernstein)

* SOLR-14155: Load all other SolrCore plugins from packages (noble)

Optimizations
---------------------
* SOLR-14975: Optimize CoreContainer.getAllCoreNames, getLoadedCoreNames and
getCoreDescriptors. (Bruno Roustant)

* SOLR-15049: Optimize same-core, same-field joins in TopLevelJoinQuery (Jason

Gerlowski)

* SOLR-14923: Indexing nested documents is faster, especially under concurrent

indexing load. In addition,
Partial updates to nested documents and Realtime Get of child documents is now
more reliable.
(David Smiley, Thomas Wöckinger)

* SOLR-15036: Automatically wrap a facet expression with a select / rollup / sort /

plist when using a
collection alias with multiple collections and count|sum|min|max|avg metrics.
(Timothy Potter)

Bug Fixes
---------------------
* SOLR-14946: Fix responseHeader being returned in response when omitHeader=true
and EmbeddedSolrServer is used
as the client (Munendra S N)

* SOLR-14940: ReplicationHandler memory leak through SolrCore.closeHooks with

unstable ZK connection. (Anver Sotnikov, Mike Drob)

* SOLR-14969: Prevent creating multiple cores with the same name which leads to
instabilities (race condition)
(Erick Erickson, Andreas Hubold)

* SOLR-14961: Fix for deleting zookeeper nodes with same path length. Only the
first zk-node was removed. (Michael Aleythe via Mike Drob)

* SOLR-14971: AtomicUpdate 'remove', 'add-distinct' operations now works on

numeric, date fields in uncommitted docs (Jason Gerlowski)

* SOLR-14983: Fix response returning original score instead of reranked score due
to query and filter combining.
(Krishan Goyal, Jason Baik, Christine Poerschke)

* SOLR-14993: Unable to download zookeeper files of 1byte in size (Erick Erickson,

Allen Sooredoo)

* SOLR-14851: Http2SolrClient doesn't handle keystore type correctly (Andras

Salamon via janhoy)

* SOLR-15009: Correctly propogate exceptions from DirectoryFactory.exists (Mike

Drob)

* SOLR-15017: Core lib directories were not being recognized unless the solrconfig
included a <lib> directive.
(Thomas Mortagne)

* SOLR-14934: Refactored duplicate "Solr Home" logic into a single place to

eliminate risk of tests using divergent values
for a single solr node. (hossman)

* SOLR-12182: Don't persist base_url in ZK as the URL scheme is variable, compute

from node_name instead when reading
state back from ZK. (Timothy Potter)

* SOLR-15046: Check if SOLR_SSL_ENABLED strictly equal to true for setting

solr.jetty.https.port (Timothy Potter)

* SOLR-14939: JSON range faceting to support cache=false parameter. (Christine

Poerschke, Mike Drob)

* SOLR-15031: Fix preventing null being wrapped in a QueryValueSource subQuery.

Such null queries can be caused by query text
resulting in an empty token stream. (Pieter van Boxtel via Mike Drob)

* SOLR-15061: Fix NPE in SearchHandler when shards.info and no QueryComponent

configured. (Bruno Roustant)

* SOLR-14571: Download speed was hard-coded in the "Replication" screen of the Solr
Admin UI. (Florin Babes via Christine Poerschke)

* SOLR-15048: Fixed collapse parser behavior when dealing with docs boosted by
QueryElevationComponent that are in the
null group to treat them consistently regardless of collapse field type or group
head selector. (hossman)

* SOLR-15058: Enforce node_name contains colon and port and find first underscore
after colon to parse context
when converting a node_name to a base URL. (Timothy Potter, Su Sasa)

* SOLR-15047: Fix collapse parser behavior when collapsing on numeric fields to

differentiate '0' group from null group (hossman)

* SOLR-10860: Return proper error code for bad input in case of inplace updates
(Tomas Eduardo Fernandez Lobbe, Munendra S N)

* SOLR-14950: Fix error in copyField regeneration when explicit source/destination

is not present in schema but
matches the dynamic rule. The copyFields are rebuilt with replace-field and
replace-field-type, if required
(Andrew Shumway, Munendra S N)

* SOLR-12539: Handle parsing of values for 'other', 'include', and 'excludeTags' in

JSON facets when specified as
comma-separated values with extra spaces (hossman, Munendra S N)

* SOLR-14514: Avoid picking 'stream' method in JSON facet when any of 'allBuckets',
'numBuckets', and 'missing' parameters are enabled
(hossman, Munendra S N)

* SOLR-12559: Fix placeholder valuesource 'FIELDNAME' not working with de-

referenced values in JSON aggregator parsing.
(hossman, Munendra S N)

* SOLR-15070: Suggester requests made with SolrJ can now use XMLResponseParser
(Jason Gerlowski)

* SOLR-15073: Fix ClassCastException in SystemInfoHandler.getSecurityInfo. (Nikolay

Ivanov, Christine Poerschke)

* SOLR-15071: Fix ArrayIndexOutOfBoundsException in contrib/ltr SolrFeatureScorer.

(Florin Babes, Ovidiu Mihalcea, David Smiley, Christine Poerschke)

Other Changes
---------------------

* SOLR-14954: Heavily edit reindexing.adoc (Sameul García Martínez and Erick

Erickson)

* SOLR-14981: Removed search results clustering contrib from 8x (due to Carrot2

incompatibility with Java 1.8).
(Dawid Weiss)

* SOLR-14995: Update Jetty to 9.4.34 (Mike Drob)

* SOLR-14958: Refactor zkHost config logic to make testing easier and reduce risk
of incorrect value being used (hossman)

* SOLR-14951: Upgrade Angular JS 1.7.9 to 1.8.0 (Kevin Risden)

* SOLR-15057: Avoid unnecessary object retention in FacetRangeProcessor. (Christine

Poerschke)

* SOLR-14999: New option for specifying the advertised Solr Port (hostPort). Still
defaults to "jetty.port". (Houston Putman)
* SOLR-15010: Thread dump using jattach if available and jstack is missing. (Eric
Pugh)

================== 8.7.0 ==================

Consult the lucene/CHANGES.txt file for additional, low level, changes in this
release.

New Features
---------------------

* SOLR-14151 Make schema components load from packages (noble)

* SOLR-14681: Introduce ability to delete .jar stored in the Package Store.

(MarcusSorealheis , Mike Drob)

* SOLR-14604: Add the ability to uninstall a package from with the Package CLI.
(MarcusSorealheis)

* SOLR-14582: Expose IWC.setMaxCommitMergeWaitMillis in Solr's index config. This

is an expert config option that can be
set when using a custom MergePolicy (doesn't have any effect on the default MP)
(Tomás Fernández Löbbe)

* SOLR-13751: Add BooleanSimilarityFactory class. (Andy Webb via Christine

Poerschke)

* SOLR-14875: Make SolrEventListeners load from packages (noble)

* SOLR-10391: ConfigSet handler now supports overrides on existing configsets.

(Tomás Fernández Löbbe)

* SOLR-11167: Avoid $SOLR_STOP_WAIT use during 'bin/solr start' if $SOLR_START_WAIT

is supplied.
(Omar Abdelnabi, Christine Poerschke)

* SOLR-14907: Support single file upload/overwrite in configSet API. (Houston

Putman)

* SOLR-14914: Add option to disable metrics collection. (ab)

* SOLR-14588: Introduce Circuit Breaker Infrastructure and a JVM heap usage memory
tracking circuit breaker implementation (Atri Sharma)

* SOLR-14615: CPU Utilization Based Circuit Breaker (Atri Sharma)

* SOLR-15052: Reducing overseer bottlenecks using per-replica states (noble, Ishan

Chattopadhyaya)

Improvements
---------------------

* SOLR-14523: Enhance gradle logging calls validation: eliminate getMessage()

(Andras Salamon via Erick Erickson)

* SOLR-14537: Improve performance of ExportWriter. (ab, Joel Bernstein)

* SOLR-14566: Request ID's ('rid') are now added by default to distributed search
requests, and can be used to correlate
 logs from the receiving coordinator node with those from downstream shard
requests. This can be disabled by providing a
disableRequestId=true request parameter. (Jason Gerlowski)

* SOLR-14635: ThreadDumpHandler has been enhanced to show lock ownership (hossman)

* SOLR-11262: Add support to stream data out using push APIs when wt=xml (yonik,
Munendra S N)

* SOLR-13205: Prevent StringIndexOutOfBoundsException when parsing field names in

SolrQueryParserBase
(pramodkumar9 via Jason Gerlowski)

* SOLR-14651: The MetricsHistoryHandler can more completely disable itself when you
tell it to.
Also, it now shuts down more thoroughly. (David Smiley)

* SOLR-14722: Track timeAllowed from earlier in the request lifecycle. (David

Smiley)

* SOLR-13438: On deleting a collection, its config set will also be deleted iff it
has been auto-created, and not used by any other collection (Anderson Dorow)

* SOLR-14802: geodist: Support most (all?) spatial field types as an argument like
LLPSF, SRPTFT, and others.
(Tom Edge, Craig Wrigglesworth)

* SOLR-14905: Update commons-io version to 2.8.0 due to security vulnerability.

(Nazerke Seidan via Bruno Roustant)

* SOLR-14691: Metrics reporting should avoid creating objects. (ab, noble)

* SOLR-14933: Ability to add T and P type replica from admin UI (Sayan Das via Eric
Pugh)

Optimizations
---------------------

* SOLR-14354: HttpShardHandler send requests in async (Cao Manh Dat).

* SOLR-14610: ReflectMapWriter to use MethodHandle instead of old reflection

(noble)

* SOLR-13132: JSON Facet perf improvements to support "sweeping" collection of

"relatedness()"
(hossman, Michael Gibney)

* SOLR-14422: Admin UI shows Angular placeholders on first load / refresh. (Colvin

Cowie via Eric Pugh)

* SOLR-14819: Fix inefficient iterator pattern in JsonSchemaValidator. (Thomas

DuBuisson via Bruno Roustant)

* SOLR-14658: SolrJ's CollectionAdminRequest.collectionStatus(collection) would

internally get
all collection statuses instead of just the specified collection. (Andy Vuong)

* SOLR-14576 : Do not use SolrCore as keys in a WeakHashMap (noble)

* SOLR-14776: Precompute fingerprint during PeerSync in parallel with fetching
fingerprint from replicas.
(Cao Manh Dat, Mike Drob via shalin)

Bug Fixes
---------------------

* SOLR-14869: ChildDocTransformer should have omitted deleted child documents. (Bar

Rotstein, David Smiley)

* SOLR-11656: TLOG replication doesn't work properly after rebalancing leaders.

(Yuki Yano via
Erick Erickson)

* SOLR-14516: Fix NPE in JSON response writer(wt=json) with /get when writing non-
stored, non-indexed docvalue field
from an uncommitted document (noble, Ishan Chattopadhyaya, Munendra S N)

* SOLR-14657: Improve error handling in IndexReader realted metrics that were

causing scary ERROR logging
if metrics were requested while Solr was in the process of closing/re-opening a
new IndexReader. (hossman)

* SOLR-14748: Fix incorrect auth/SSL startup logging (Jason Gerlowski)

* SOLR-14751: Zookeeper Admin screen not working for old ZK versions (janhoy)

* SOLR-14677: Improve DIH termination logic to close all DataSources,

EntityProcessors (Jason Gerlowski)

* SOLR-14703: Fix edismax replacement of all whitespace characters with spaces

(Yuriy Koval via Jason Gerlowski)

* SOLR-14700: Avoid NullPointerException in TupleStream.getShards() when

streamContext is null.
(Mads Bondo Dydensborg, Christine Poerschke, Mike Drob)

* SOLR-14752: Fix error in Zookeeper status when Prometheus plugin is enabled in ZK

(Philipp Trulson via janhoy)

* SOLR-14774: HealthCheckHandler is no longer an implicit SolrCore handler and can

be configured from solr.xml
(Tomás Fernándex Löbbe)

* SOLR-14714: Solr.cmd in windows loads the incorrect jetty module when using
java>=9 (Endika Posadas via
Erick Erickson)

* SOLR-14821: {!terms} docValuesTermsFilterTopLevel method now works with single-

valued strings (Anatolii Siuniaev
via Jason Gerlowski)

* SOLR-13181: macro expansion of parameters could result in a

StringIndexOutOfBoundsException.
Also, params with macros will be processed more strictly by LTR FeatureWeight.
(David Smiley, Cesar Rodriguez, Christine Poerschke)

* SOLR-14503: Use specified waitForZk value as connection timeout for zookeeper in

SolrDispatcherFilter.
 Also, consume specified SOLR_WAIT_FOR_ZK in bin/solr.cmd (Colvin Cowie via
Munendra S N)

* SOLR-14936: Fixed Grafana dashboard filters for collection, shard, replica and
core (janhoy)

* SOLR-14924: Some ReplicationHandler metrics are reported using incorrect types.

(ab)

* SOLR-14483: Fix empty drop down for adding replica in Admin UI (Sayan Das via
Eric Pugh)

* SOLR-14549: Fix listing of Files in a Directory on Solr Admin UI (Kevin Risden)

Other Changes
---------------------

* SOLR-14592: Upgrade Zookeeper to 3.6.1. NOTE: this required upgrading netty to

4.1.50 (Erick Erickson)

* SOLR-10742: SolrCores.getNamesForCore is quite inefficient and blocks other core

operations.
NOTE: this experimental method has been removed (Erick Erickson)

* SOLR-13939: Extract any non-gradle related patches (deprecations, URL fixes,

etc.) from gradle effort. NOTE:
this will be in several separate commits/pushes. (Erick Erickson)

* SOLR-14637: Update CloudSolrClient examples to remove deprecated method. (Andras

Salamon via Eric Pugh)

* SOLR-14676: Update commons-collections to 4.4 and use it in Solr (Erick Erickson)

* SOLR-11868: Deprecate CloudSolrClient.setIdField, use information from Zookeeper

(Erick Erickson)

* SOLR-14641: PeerSync, remove canHandleVersionRanges check (Cao Manh Dat)

* SOLR-14731: Rename @SolrSingleThreaded to @SolrThreadUnsafe, mark

DistribPackageStore with the annotation
(marcussorealheis)

* SOLR-14579: Comment SolrJ 'Utils' generic map functions

(Megan Carey and a lot of education from Uwe Schindler via Erick Erickson)

* SOLR-10471: Update default Zookeeper session timeout to 30s in bin/solr and

bin/solr.cmd
(Michael Braun, Markus Jelsma via Munendra S N)

* SOLR-14876: Upgrade to zookeeper 3.6.2 (odidev via Erick Erickson)

* SOLR-14333: Implement toString in Collapse filter so that proper parsed queries

returned in debug response.
Also, Deprecate unused constants NULL_COLLAPSE, NULL_IGNORE, NULL_EXPAND,
HINT_MULTI_DOCVALUES in collapse parser.
(Guna Sekhar Dora Kovvuru, Munendra S N, Mike Drob)

* SOLR-12987: Deprecated plugins/features are now logged once and with log category
org.apache.solr.DEPRECATED
 (David Smiley)

* SOLR-14910: Use in-line tags for logger declarations in Gradle ValidateLogCalls

that are non-standard,
change //logok to //nowarn (Erick Erickson)

* SOLR-14659: Remove restlet as dependency for the ManagedResource API (Timothy

Potter, noble)

================== 8.6.3 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Bug Fixes
---------------------
* SOLR-14859: DateRangeField now throws errors when invalid field/fieldType options
specified; no longer silently
accepts incompatible option values (Jason Gerlowski, Chris Hostetter, Munendra S
N)

* SOLR-14768: Fix HTTP multipart POST to Solr -- a regression from 8.6.0. Many
Jetty classes are not classpath-visible
from the Solr webapp. (David Smiley)

* SOLR-14850: Fix ExactStatsCache NullPointerException when shards.tolerant=true.

(Yevhen Tienkaiev via ab)

* SOLR-14897: Fix unlimited number of forwarding the request from one node to
another node. (hossman, Munendra S N)

* SOLR-14898: Stop returning duplicate HTTP response headers when requests are
forward to another node. (hossman)

================== 8.6.2 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Bug Fixes
---------------------
* SOLR-14751: Zookeeper Admin screen not working for old ZK versions (janhoy)

* SOLR-14663: Copy ConfigSet root data from base ConfigSet when using CREATE
command (Andras Salamon, Tomás Fernández Löbbe)

================== 8.6.1 ==================

Bug Fixes
---------------------

* SOLR-14665, SOLR-14706: Revert SOLR-12845 adding of default autoscaling cluster

policy, due to performance
issues. (Ishan Chattopadhyaya, Houston Putman, Gus Heck, ab)

* SOLR-14671: Parsing dynamic ZK config sometimes cause NuberFormatException

(janhoy)

================== 8.6.0 ==================

Consult the lucene/CHANGES.txt file for additional, low level, changes in this
release.

New Features
---------------------
* SOLR-14210: HealthCheckHandler can now require that all cores are healthy before
returning 'OK'
To enable, add &requireHealthyCores=true to enable (janhoy, Houston Putman,
shalin)

* SOLR-13942: A read API at /api/cluster/zk/* to fetch raw ZK data and view

contents of a ZK directory (noble)

* SOLR-14237: A new panel with security info in admin UI's dashboard (Ishan
Chattopadhyaya, Moshe Bla)

* SOLR-12131: ExternalRoleRuleBasedAuthorizationPlugin which gets user's roles from

request (janhoy)

* SOLR-14478: Allow the diff Stream Evaluator to operate on the rows of a matrix
(Joel Bernstein)

* SOLR-14476: Add percentiles and standard deviation aggregations to stats, facet

and
timeseries Streaming Expressions (Joel Bernstein)

* SOLR-14470: Add streaming expressions to /export handler. (ab, Joel Bernstein)

* SOLR-14404: CoreContainer level custom requesthandlers (noble)

* SOLR-13749: Join queries can now work cross-collection, even when sharded or when
spanning nodes.
(Dan Fox, Kevin Watters, via Gus Heck, David Smiley)

* SOLR-14481: Add drill Streaming Expression for efficient and accurate high
cardinality aggregation.
(Joel Bernstein)

* SOLR-14599: Package manager support for cluster level plugins (see SOLR-14404)
(Ishan Chattopadhyaya)

* SOLR-14590: Add support for RankFields. RankFields allow the use of per-document
scoring factors in a
way that lets Solr skip over non-competitive documents when ranking. See SOLR-
13289.
(Tomás Fernández Löbbe, Varun Thacker)

Improvements
---------------------
* SOLR-14316: Remove unchecked type conversion warning in JavaBinCodec's
readMapEntry's equals() method
(Aroop Ganguly, Tomás Fernández Löbbe, Noble Paul, Anshum Gupta)

* SOLR-14260: Make SolrJ ConnectionSocketFactory pluggable via

SocketFactoryRegistryProvider setting on HttpClientUtil
(Andy Throgmorton via David Smiley)

* SOLR-14342: Load cores in an order that makes collections available sooner and
reduces leaderVoteWait timeouts in
large SolrCloud clusters. (David Smiley)

* SOLR-14329: Add support to choose collapse group to expand in ExpandComponent

based on cost (Munendra S N)

* SOLR-14307: User defined "<cache/>" entries in solrconfig.xml now support

enabled="true|false" just like
core searcher caches. (hossman)

* SOLR-14364: LTR's SolrFeature "fq" now supports PostFilters (e.g. collapse).

(David Smiley)

* SOLR-14396: TaggerRequestHandler should not error on an empty index (Trey

Grainger)

* SOLR-12845: Add a default autoscaling cluster policy. (shalin, varunthacker, ab)

* SOLR-14433: Metrics: SolrShardReporter's default metrics list now includes TLOG

and UPDATE./update (David Smiley)

* SOLR-14423: Move static SolrClientCache from StreamHandler to CoreContainer for

wider reuse and better life-cycle management. (ab)

* SOLR-14407: Handle shards.purpose in the postlogs tool (Joel Bernstein)

* SOLR-13325: ComputePlanAction now supports a collection selector of the form

`collections: {policy: my_policy}`
which can be used to select multiple collections that match collection
property/value pairs. This is useful to
maintain a whitelist of collections for which actions are taken without needing
to hard code the collection names
themselves. The collection hints are pushed down to the policy engine so
operations for non-matching collections
are not computed at all. (ab, shalin)

* SOLR-14419: json.queries as well as other parameters might be referred via

{"param":"ref"} in Query DSL (Mikhail Khludnev)

* SOLR-11334: hl.fl and tv.fl now parse field lists when they have both commas and
spaces
(David Smiley, Yasufumi Mizoguchi)

* SOLR-14442: bin/solr and bin\solr.cmd invoke jstack <SOLR_PID> before forceful

termination, if jstack is available.
Also, bin\solr.cmd executes forceful termination even port is unbinded already
(Christine Poerschke via Mikhail Khludnev).

* SOLR-14384: SolrRequestInfo now stacks internally when a new request is

set/clear'ed.
Also fixes SolrIndexSearcher.warm which should have re-instated previous SRI.
(Nazerke Seidan, David Smiley)

* SOLR-14561: CoreAdminAPI's parameters instanceDir and dataDir are now validated,

and must be relative to either
SOLR_HOME, SOLR_DATA_HOME or coreRootDir. Added new solr.xml config 'allowPaths',
controlled by system property
'solr.allowPaths' that allows you to add other allowed paths when needed.
* SOLR-13286: Metrics will no longer write a (sometimes large) log message every
minute. This can be re-enabled
via log4j2.xml if desired, or other HttpSolrCall log messages may be quieted on a
per handler basis. (Gus Heck)

* SOLR-14539: Introducing {!bool excludeTags=...} for Query DSL. (Mikhail Khludnev)

Optimizations
---------------------
* SOLR-8306: Do not collect expand documents when expand.rows=0 (Marshall Sanders,
Amelia Henderson)

* SOLR-14340: Remove unnecessary configset verification checks when SolrCloud

merely wants to know which configset a
collection is using. Improves CLUSTERSTATUS times for massive clusters.
(Mathieu Marie, David Smiley)

* SOLR-14365: CollapsingQParser - Avoiding always allocate int[] and float[] with

size equals to number of unique values.
(Cao Manh Dat)

* SOLR-14376: Optimize filter queries that match all docs. (David Smiley)

* LUCENE-7788: fail precommit on unparameterised log messages and examine for

wasted work/objects (Erick Erickson)

* SOLR-13289: When the "minExactCount" parameters is provided in queries and it's

value is lower than the number of hits,
Solr can speedup the query resolution by using the Block-Max WAND algorithm (see
LUCENE-8135). When doing this, the
value of matching documents in the response (numFound) will be an approximation.
(Ishan Chattopadhyaya, Munendra S N, Tomás Fernández Löbbe, David Smiley)

* SOLR-14472: Autoscaling "cores" preference now retrieves the core count more
efficiently, and counts all cores.
(David Smiley)

* SOLR-14552: Add BlockMax-WAND support to ReRank queries (Tomás Fernández Löbbe)

* SOLR-14554: Add BlockMax-WAND support for queries where the score is requested
(Tomás Fernández Löbbe)

* SOLR-14634: Limit the HTTP security headers to "/solr" end point (noble)

Bug Fixes
---------------------
* SOLR-13264: IndexSizeTrigger aboveOp / belowOp properties not in valid
properties.
(Christine Poerschke, ab)

* SOLR-13199: Return proper error when invalid parentFilter is passed in

ChildDocTransformer
(Johannes Kloos, Munendra S N, David Smiley, Mikhail Khludnev)

* SOLR-14347: Autoscaling placement wrong when concurrent replica placements are

calculated. (ab)

* SOLR-14128: Improve distributed locking around managed schema upgrade process.

(ab)
* SOLR-14302: Ensure Solr always includes the stacktrace for exceptions by using '-
OmitStackTraceInFastThrow'
(hossman)

* SOLR-14317: HttpClusterStateProvider throws exception when only one node down.

(Lyle Wang via Ishan Chattopadhyaya)

* SOLR-14356: PeerSync should not fail with SocketTimeoutException from hanging

nodes (Cao Manh Dat)

* SOLR-14402: Avoid creating new exceptions for every request made to

MDCAwareThreadPoolExecutor by distributed search.
This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which
fixed distributed updates but not
distributed search. (shalin)

* SOLR-14291: Handle dotted fields in legacy Analytics Component (Anatolii Siuniaev

via Mikhail Khludnev)

* SOLR-14371: Zk StatusHandler now parse dynamic zk server config if supported,

fixing Admin UI Zookeeper Status
screen in case dynamic reconfig host list differs from static zkHost string
(janhoy)

* SOLR-14421: New examples in solr.in.cmd in Solr 8.5 don't work as provided

(Colvin Cowie via janhoy)

* SOLR-14431: SegmentsInfoRequestHandler does not release IndexWriter (Tiziano

Degaetano, ab)

* SOLR-14463: Solr Admin ZkStatus page now works with ZK 3.6, without 'For input
string: "null"' error (janhoy, Bernd Wahlen)

* SOLR-14456: Fix Content-Type header usage when a request is forwarded from Solr
node to Solr
node with compression enabled (samuelgmartinez via Houston Putman)

* SOLR-8394: /admin/luke was always showing 0 for indexHeapUsageBytes. It should

work now.
(Steve Molloy, Isabelle Giguere, David Smiley)

* SOLR-14492: Fix ArrayIndexOutOfBoundsException in json.facet 'terms' when

FacetFieldProcessorByHashDV is
used with aggregations over multivalued numeric fields (hossman)

* SOLR-14477: Fix incorrect 'relatedness()' calculations in json.facet 'terms' when

'prefix' option is used
(hossman)

* SOLR-14504: ZkController LiveNodesListener has NullPointerException in startup

race.
(Colvin Cowie via ab)

* SOLR-14498: Upgrade to Caffeine 2.8.4, which fixes the cache poisoning issue.
(Jakub Zytka, ab)

* SOLR-14517: Dont ignore 'mm' localparam on edismax queries using operators (Yuriy
Koval via Jason Gerlowski)
* SOLR-14491: Intercepting internode requests in KerberosPlugin when HTTP/2 client
is used (Ishan Chattopadhyaya, Moshe Bla)

* SOLR-14525: SolrCoreAware, ResourceLoaderAware should be honored for plugin

loaded from packages (noble)

* SOLR-9679: When removing zk node /security.json, security is now disabled

gracefully (janhoy)

* SOLR-14520: Fixed server errors from the json.facet allBuckets:true option when
combined with refine:true
(Michael Gibney, hossman)

* SOLR-14467: Fix relatedness() stat in json.facets to no longer cause server

errors (or nonsense results)
when combined with allBuckets:true. (Michael Gibney, hossman)

* SOLR-13203: Return 400 status code on invalid dynamic field for Edismax's user
Fields
(Johannes Kloos, mrsoong via Munendra S N)

* SOLR-14550: Fix duplicates issue in Atomic updates with add-distinct (Thomas

Corthals, Munendra S N)

* SOLR-14345: Return proper error message when non-BinaryResponseParser is used in

solrJ (Munendra S N)

* SOLR-14516: NPE in JsonTextWriter (noble)

* SOLR-14577: Return 400 BAD REQUEST when field is missing on a Terms query parser
request
(Tomás Fernández Löbbe)

* SOLR-14409: Existing violations allow bypassing policy rules when adding new
replicas (noble, ab)

* SOLR-14584: Correct SOLR_SSL_KEY_STORE and SOLR_SSL_TRUST_STORE example comments

in solr.in.sh and solr.in.cmd files
(Aren Cambre via Christine Poerschke)

* SOLR-14628: hl.fragsizeIsMinimum now defaults to true, thus fragments size like

previous to Solr 8.5 and addresses a
significant performance regression. (David Smiley, Michal Hlavac)

Other Changes
---------------------
* SOLR-14197: SolrResourceLoader: marked many methods as deprecated, and in some
cases rerouted exiting logic to avoid
them. (David Smiley)

* SOLR-14312: Upgrade Zookeeper to 3.5.7 (Erick Erickson)

* SOLR-14296: Update netty to 4.1.46 (Andras Solaman via Erick Erickson)

* SOLR-10157: Improve error messages when unknown aggregations are specified in the
request (hossman, Munendra S N)

* SOLR-13944: Remove redundant checks in SpellCheckCollator and add tests for

spellcheck with collapse
(Stefan, Munendra S N, Tomas Eduardo Fernandez Lobbe)

* SOLR-12353: SolrDispatchFilter expensive non-conditional debug line degrades

performance (Pascal Proulx via Erick Erickson)

* SOLR-13842: Remove redundant defaults from ImplicitPlugins.json (Munendra S N)

* SOLR-13893: BlobRepository reads max jar size from `runtime.lib.size` system

property. Old `runtme.lib.size` is Deprecated
(Erick Erickson, Kesharee Nandan Vishwakarma, Munendra S N)

* SOLR-14367: Upgrade Tika to 1.24 (Mibo via Erick Erickson)

* SOLR-14351: Hardened MDC logging; sometimes info was absent (David Smiley)

* SOLR-14386: Update Jetty to 9.4.27 and dropwizard-metrics version to 4.1.5 (Erick

Erickson)

* SOLR-9909: SolrjNamedThreadFactory is deprecated in favor of

SolrNamedThreadFactory. DefaultSolrThreadFactory is
removed from solr-core in favor of SolrNamedThreadFactory in solrj package and
all solr-core classes now use
SolrNamedThreadFactory. (Andras Salamon, shalin)

* SOLR-14391: Removed internal-ish ScoreFilter marker interface; only used by {!

collapse}. Removed needless code in
SolrIndexSearcher.getDocSet(List<Query>) (processes some filter queries). (David
Smiley)

* SOLR-13886: HDFSSyncSliceTest and SyncSliceTest started failing frequently (Kevin

Risden)

* SOLR-14173: Major redesign of the Solr Reference Guide (Cassandra Targett)

* SOLR-14461: Replaced commons-fileupload dependency with using Jetty's facilities.

(David Smiley)

* SOLR-14466: Upgrade log4j2 to latest release (2.13.2) (Erick Erickson)

* SOLR-11934: Visit Solr logging, it's too noisy. Note particularly that the
messages for
opening a new searcher have changed and include the autowarm time. (Erick
Erickson)

* SOLR-7880: Update commons-cli to 1.4 (Erick Erickson)

* SOLR-14226: Fix or suppress 14 resource leak warnings in apache/solr/core (Andras

Salaman via
Erick Erickson)

* SOLR-14482: Fix or suppress warnings in solr/search/facet (Erick Erickson)

* SOLR-14485: Fix or suppress 11 resource leak warnings in apache/solr/cloud

(Andras Salaman via
Erick Erickson)

* SOLR-14495: Fix or suppress warnings in solr/search/function (Erick Erickson)

* SOLR-14280: Improve error reporting in SolrConfig (Andras Salamon via Jason
Gerlowski)

* SOLR-14474: Fix remaining auxilliary class warnings in Solr (Erick Erickson)

* SOLR-14519: Fix or suppress warnings in solr/cloud/autoscaling/ (Erick Erickson)

* SOLR-14526: fix or suppress warnings in apache/solr/core (Erick Erickson)

* SOLR-14533: Fix or suppress warnings in solr/handler/admin (Andras Salamon, Erick

Erickson)

* SOLR-14535: Fix or suppress warnings in apache/solr/handler/component (Erick

Erickson)

* SOLR-14538: Fix or suppress remaining warnings in apache/solr/handler (Erick

Erickson)

* SOLR-14536: Fix or suppress warnings in apache/solr/common (Erick Erickson)

* SOLR-14480: Fix or suppress warnings in solr/cloud/api (Erick Erickson)

* SOLR-13492: Ensure explicit GCs are concurrent by adding

'+ExplicitGCInvokesConcurrent'.
(Guna Sekhar Dora, Shawn Heisey, Munendra S N)

* SOLR-14542: Fix or suppress warnings in solr/handler/dataimport (Erick Erickson)

* SOLR-14544: Fix or suppress warnings in solr/client/solrj/io/eval (Erick

Erickson)

* SOLR-14543: Fix or suppress warnings in apache/solr/search (Erick Erickson)

* SOLR-14545: Fix or suppress warnings in apache/solr/update (Erick Erickson)

* SOLR-14548: Address warning: static member should be qualified by type name (Mike
Drob)

* SOLR-14547: Fix or suppress warnings in solr/client/solrj/io/stream (Erick

Erickson)

* SOLR-14455: Fix or suppress warnings in solr/test-framework (Erick Erickson)

* SOLR-14559: Fix or suppress warnings in solr/core/src/java/org/apache/solr/util,

response, cloud, security, schema, api (Erick Erickson)

* SOLR-14563: Fix or suppress warnings in solr/contrib (Erick Erickson)

* SOLR-14565: Fix or suppress warnings in solrj/impl and solrj/io/graph (Erick

Erickson)

* SOLR-14564: Fix or suppress remaining warnings in solr/core (Erick Erickson)

* SOLR-14567: Fix or suppress remaining warnings in solrj (Erick Erickson)

* SOLR-14556: Fix or suppress warnings in solrj/cloud/autoscaling (Erick Erickson)

* SOLR-14573: Fix or suppress warnings in solrj/src/test (Erick Erickson)

* SOLR-14574: Fix or suppress warnings in solr/core/src/test (parts 1 and 2) (Erick
Erickson)

* SOLR-13268: Clean up any test failures resulting from defaulting to async logging
(Erick Erickson)

* LUCENE-9411: Fail complation on warnings, 9x gradle-only (Erick Erickson, Dawid

Weiss)
Only mentioned in 8.6 because I backported some more warning suppressions but not
the fail-on-warnings.

* SOLR-14589: Remove IntelliJ //noinspection comments (Erick Erickson)

* SOLR-14541: Ensure classes that implement equals implement hashCode or suppress

warnings
(gezapeti, Ilan Ginsburg, Erick Erickson)

* SOLR-14022: Deprecate CDCR (Joel Bernstein, Ishan Chattopadhyaya)

* SOLR-14066: Data Import Handler is deprecated. It is scheduled to be removed as

of 9.0 and a community supported
package for the same may now be used instead. (Ishan Chattopadhyaya, janhoy)

* SOLR-14021: HDFS support is deprecated. It is scheduled to be removed as of 9.0.

Please see JIRA for more details.
(Joel Bernstein, Ishan Chattopadhyaya)

* SOLR-14603: Upgrade Restlet to 2.4.3 (marcussorealheis)

================== 8.5.2 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Bug Fixes
---------------------

* SOLR-14411: Fix Admin UI collection/core drop-downs placeholder text. Completes

work started in SOLR-14359 (janhoy)

* SOLR-14471: Fix bug in shards.preference behavior, base replica selection

strategy not applied to the last group of
equivalent replicas. (Michael Gibney via Tomás Fernández Löbbe)

================== 8.5.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Bug Fixes
---------------------

* SOLR-14359: Admin UI collection/core drop-downs had wrong placeholder text. Fixed

by upgrading angular-chosen
This fix was incomplete, see SOLR-14411 for the followup (janhoy)

================== 8.5.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

New Features
---------------------
* SOLR-12490: Introducing json.queries in JSON Request API. Every property of this
object holds one or many named
Query DSL queries. It's optional and doesn't impact response without explicit
referencing these queries by names
(Anatolii Siuniaev via Mikhail Khludnev)

* SOLR-14130: Add postlogs command line tool for indexing Solr logs (Joel
Bernstein)

* SOLR-13749: New cross collection join filter (XCJF) (Dan Fox, Kevin Watters, via
Gus Heck)

* SOLR-12325: Introducing uniqueBlock({!v=type:parent}) aggregation (Anatolii

Siuniaev via Mikhail Khludnev)

* SOLR-14241: New delete() Stream Decorator (hossman)

* SOLR-14139: Support backtick phrase queries in Streaming Expressions (Joel

Bernstein)

* SOLR-12238: Synonyms can be individually boosted by marking them with '|<boost>'

in
the synonyms file, and adding a DelimitedBoostTokenFilter to the analysis chain
(Alessandro Benedetti, Alan Woodward)

Improvements
---------------------
* SOLR-14120: Define JavaScript methods 'includes' and 'startsWith' to ensure
AdminUI can be displayed when using
Internet Explorer 11 (jafurrer).

* SOLR-14042: Fix varargs precommit warnings (Andraas Salamon via Jason Gerlowski)

* SOLR-14095: Replace Java serialization with Javabin in the Overseer queues (Tomás
Fernández Löbbe)

* SOLR-14125: Make <expressible> plugins work with packages (noble)

* SOLR-14138: Jetty's NCSA-formatted RequestLog can be enabled via environment

variable,
SOLR_REQUESTLOG_ENABLED=true. This will respect SOLR_LOGS_DIR if it is set.
(rmuir)

* SOLR-14136: Jetty's InetAccessHandler can be enabled via environment variables,

SOLR_IP_WHITELIST and SOLR_IP_BLACKLIST. These variables can restrict access to
Solr based on IP addresses/networks. (rmuir)

* SOLR-14131: Add maxQueryLength option to DirectSolrSpellchecker. (Andy Webb via

Bruno Roustant)

* SOLR-13984: Java's SecurityManager sandbox can be enabled via environment

variable,
SOLR_SECURITY_MANAGER_ENABLED=true. (rmuir)

* SOLR-14129: Reuse Jackson ObjectMapper in AuditLoggerPlugin (janhoy)

* LUCENE-9093: The Unified highlighter has two new passage sizing parameters,
hl.fragAlignRatio and
hl.fragsizeIsMinimum, with defaults that aim to better center matches in
fragments than previously. See the ref guide.
Regardless of the settings, the passages may be sized differently than before.
(Nándor Mátravölgyi, David Smiley)

* SOLR-14154: Return correct isolation level when retrieving it from the SQL
Connection (Nick Vercammen, Kevin Risden)

* SOLR-14132: Upgrade Angular JS 1.3.8 to 1.7.9 (Kevin Risden)

* SOLR-14221: Upgrade restlet to version 2.4.0 (janhoy)

* SOLR-10567: Add Support for DateRangeField in JSON Facet range (Stephen Weiss,
Munendra S N)

* SOLR-14245: Validate Replica / ReplicaInfo on creation. (ab)

* SOLR-14194: Highlighting now works when the uniqueKey field is not stored but has
docValues. And the original
highlighter can now highlight text fields from docValues. (Andrzej Wislowski,
David Smiley)

* SOLR-14270: export command to have an option to write to a zip file (noble)

* SOLR-14286: Upgrade Jaegar to 1.1.0 (Cao Manh Dat)

* SOLR-14281: Make sharedLib configurable through system property. Also allow

multiple paths, separated by comma.
See Reference Guide 'format-of-solr-xml.adoc' and comment in 'solr.in.sh' for
details (janhoy)

* SOLR-13892: New "top-level" docValues join implementation (Jason Gerlowski, Joel

Bernstein)

* SOLR-14242: HdfsDirectory now supports indexing geo-points, ranges or shapes.

(Adrien Grand)

* SOLR-13965: In GraphHandler, support <expressible> configuration and deprecate

<streamFunctions> configuration.
(Eric Pugh via Christine Poerschke)

Optimizations
---------------------

* SOLR-13890: Add "top-level" DV implementation for {!terms} queries. This

approach tends to be more efficient for
queries with very large numbers of terms. The new implementation is used by
default for method=docValuesTermsFilter
terms queries that are searching 500 or more terms. Users wishing to ignore this
heuristic can choose the
docValuesTermsFilterTopLevel or docValuesTermsFilterPerSegment methods directly.
(Jason Gerlowski, Joel Bernstein, David Smiley)

* SOLR-13808: filter in BoolQParser and {"bool":{"filter":..}} in Query DSL are

cached by default (Mikhail Khludnev)
* SOLR-14040: The shareSchema solr.xml option now supports SolrCloud. shareSchema
is useful when you have many
cores per node and large schemas. (David Smiley)

Bug Fixes
---------------------

* SOLR-14099: Fixed @LogLevel annotation in test-framework to correctly 'unset'

Loggers after test (hossman)

* SOLR-14017: Ensure bin/solr's -q and -v options work with examples (Andy Vuong
via Jason Gerlowski)

* SOLR-14122: SimUtils converts v2 to v1 request params incorrectly. (Li Cao, ab)

* SOLR-13089: Fix lsof edge cases in the solr CLI script (Martijn Koster via
janhoy)

* SOLR-14198: Nullpointer exception in AuditEvent when initialized from

AuthorizationContext (janhoy)

* SOLR-6613: TextField.analyzeMultiTerm does not throw an exception when Analyzer

returns no terms. (Bruno Roustant)

* SOLR-14192: Race condition between SchemaManager and ZkIndexSchemaReader. (ab)

* SOLR-12859: Fixed DocExpirationUpdateProcessorFactory to work with BasicAuth and

other auth plugins
that delegate to PKI for server initiated node-to-node communication. (hossman)

* SOLR-11746: Adding existence queries for PointFields. DocValuesFieldExistsQuery

and NormsFieldExistsQuery are used for existence queries when possible. (Houston
Putman, hossman, Kai Chan)

* SOLR-14207: Fix logging statements with less or more arguments than placeholders.
(shalin)

* SOLR-14172: Collection metadata remains in zookeeper if too many shards are

requested. This also fixes a bug where
an inability to assign a node based on existing autoscaling policy resulted in a
server error instead of a bad request.
(Andras Salamon, Kevin Risden, shalin)

* SOLR-14196: AdminUI login not working for JWTAuth when blockUnknown=false

(janhoy)

* SOLR-14205: Do not fail when given timeout to connectionImpl.isValid() = 0 (Nick

Vercammen via Kevin Risden)

* SOLR-11554: Support handling OPTIONS request for Hadoop authentication filter

(Peter Cseh via Kevin Risden)

* SOLR-14189: Switch from String.trim() to StringUtils.isBlank() in query parsers

(Andy Webb via Uwe Schindler)

* SOLR-13897: Fix unsafe publication of Terms object in ZkShardTerms that can cause
visibility issues
and race conditions under contention. (shalin)
* SOLR-14090: Handle the case in `delete-copy-field` when source is a dynamic field
(Frank Iversen, Munendra S N)

* SOLR-14219: SOLR-14095 Introduced an issue for rolling restarts (Incompatible

Java serialization). This change
Fixes the compatibility issue while keeping the functionality in SOLR-14095.
(Andy Webb, Tomás Fernández Löbbe)

* SOLR-13887: Use default idleTimeout instead of 0 for HTTP2 requests. (Houston

Putman)

* SOLR-14038: Admin UI display for "state.json" should be in a scollable region

(Kevin Risden)

* SOLR-14058: Fix IndexOutOfBoundsException in PeerSync that can prevent nodes from

recovering
under certain circumstances. (yonik)

* SOLR-13041: Add hashCode for autoscaling.Condition to accompany the already

present equals.
(Zsolt Gyulavari via Christine Poerschke)

* SOLR-12550: ConcurrentUpdateSolrClient doesn't respect timeouts for commits and

optimize.
(Marc A. Morissette, Bérénice MAUREL via shalin)

* SOLR-14250: Do not log error when trying to consume non-existing input stream due
to Expect: 100-continue (janhoy)

* SOLR-13411: Deny atomic update for route.field, uniqueKey, version and throw
exception. (Dr Oleg Savrasov via Mikhail Khludnev)

* SOLR-14252: Avoid NullPointerException in AggregateMetric. (Andy Webb via ab)

* SOLR-14287: Admin UI Properties screen does not show colons (janhoy)

* SOLR-14299: IndexFetcher doesn't reset error count to 0 after the last packet is
received (Pushkar Raste via Mike Drob)

* SOLR-14073: Fix segment look ahead NPE in CollapsingQParserPlugin (Joel

Bernstein)

Other Changes
---------------------

* SOLR-14047: Make sure tests don't pickup other Hadoop installs (Kevin Risden)

* SOLR-14054: Upgrade to Tika 1.23 when available (Tim Allison)

* SOLR-14091: Remove deprecated soLingerTime when configuring Jetty connector

(Matthias Krueger via Kevin Risden)

* SOLR-13778: Solrj client will retry requests on SSLException with a suppressed

SocketException
(very likely a hard-closed socket connection).

* SOLR-14169: Fix 20 Resource Leak warnings in SolrJ's apache/solr/common (Andras

Salamon via Tomás Fernández Löbbe)
* SOLR-14184: Internal 'test' variable DirectUpdateHandler2.commitOnClose has been
removed and replaced with
TestInjection.skipIndexWriterCommitOnClose. Users that modified
DUH2.commitOnClose in test cases for custom
plugins/modicitations should now use TestInjection instead. (hossman)

* SOLR-14248: Improve ClusterStateMockUtil and make its methods public. (shalin)

* SOLR-14209: Upgrade JQuery to 3.4.1 (Kevin Risden)

* SOLR-13996: Refactor HttpShardHandler.prepDistributed method. (shalin)

* SOLR-13794: Replace redundent test only copy of '_default' configset with

SolrTestCase logic to correctly
set 'solr.default.confdir' system property (hossman)

* SOLR-14263: Update jvm-settings.adoc (Erick Erickson)

* SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches
during SolrCore reload. Removed Bandaid kludge. (Erick Erickson)

* SOLR-10306: Document in Reference Guide how to disable or reduce swapping

(janhoy)

* SOLR-13983: Process execution is removed from SystemInfoHandler. A best-effort

attempt to
execute `uname -a` and "`uptime`" on non-Windows platforms is no longer made.
(rmuir)

* SOLR-14114: Add WARN to Solr log that embedded ZK is not supported in production
(janhoy)

================== 8.4.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.5.5
Jetty 9.4.19.v20190610

Upgrade Notes
---------------------

* SOLR-14158: Package manager now stores keys in package store instead of ZK. If
you're using package manager and have already
added any repositories, then you would need to re-add all the keys from /keys/exe
location, one-by-one, using:
"bin/solr add-key <keyfile.der>"

Bug Fixes
----------------------

* SOLR-14106: Cleanup Jetty SslContextFactory usage (Ryan Rockenbaugh, Jan Hoydahl,

Kevin Risden)
* SOLR-14109: Always log to stdout from server/scripts/cloud-scripts/zkcli.{bat|sh}
(janhoy)

* SOLR-14165: SolrResponse serialVersionUID has changed in a backward incompatible

way (Andy Webb via noble)

* SOLR-14163: SOLR_SSL_CLIENT_HOSTNAME_VERIFICATION needs to work with Jetty

server/client SSL contexts (Kevin Risden)

Improvements
---------------------

* SOLR-14158: Package manager to store public keys in a special "trusted" location

instead of in ZooKeeper
(noble, Ishan Chattopadhyaya)

================== 8.4.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.5.5
Jetty 9.4.19.v20190610

Upgrade Notes
---------------------

* SOLR-13823:
org.apache.solr.search.grouping.distributed.command.QueryCommand.Builder has new
method 'setMainQuery' which is used
to set top-level query. build() would fail if called without setting mainQuery.

* SOLR-13817: Deprecate legacy SolrCache implementations. Users are encouraged to

transition their
configurations to use org.apache.solr.search.CaffeineCache instead. (ab)

* SOLR-13982: Some security-related http headers such as Content-Security-Policy

are now set. If you have custom html served
up by Solr's http server that contains inline javascript, it will no longer
execute in modern browsers. You can fix your JS
code to not run inline anymore, or edit etc/jetty.xml and weaken the CSP, or
remove/alter the headers with a reverse proxy. (rmuir)

* SOLR-13806: QueryResponse.getExplainMap() type has changed from Map<String,

String> to Map<String, Object>
in order to accommodate structured explanations. This is a partially back-
compatible change - compiled
third-party components will work the same as before due to type erasure but
source code changes may be
required.

* SOLR-14025: VelocityResponseWriter has been hardened - only trusted configsets

can render configset provided
 templates and rendering templates from request parameters has been removed.

* SOLR-13904: timeAllowed parameter is allowed to have 0 value (Houston Putman,

Mikhail Khludnev)

* SOLR-13970: Using Collapse filter or expand component with grouping is explicitly

disallowed as the combination
would cause inconsistent behavior and NPEs.

* SOLR-13978: The default configset no longer has the following:

- Library inclusions (<lib ../>) for extraction, solr-cell libs, clustering,
velocity, LTR and language identifier
- /browse, /tvrh and /update/extract handlers
- TermVector component (if someone wants it, can be added using config APIs)
- XSLT response writer
- Velocity response writer
If you want to use them in your collections, please add them to your configset
manually or through the Config APIs.

* SOLR-14071: Untrusted configsets (ones that are uploaded via unsecured configset
API) cannot use <lib> directive.
Consider enabling authentication/authorization so that the uploaded configsets
are trusted.
Note: If you already have a collection using untrusted configset that uses <lib>
directive, it will not load after
upgrading to 8.4. You can re-upload your configset using "bin/solr zk -
upconfig .." or place your libraries in the
classpath and restart Solr.

* SOLR-14065: VelocityResponseWriter has been deprecated and may be removed in a

future version.

* SOLR-14072: The "Blob Store" API and "runtimeLib" plugin mechanism that uses it
is now considered deprecated. The
replacement to it is the "Package Management" system, which includes a "File
Store". These are experimental currently
but will grow/stabalize/mature.

New Features
---------------------
* SOLR-13821: A Package store to store and load package artifacts (noble, Ishan
Chattopadhyaya)

* SOLR-13822: A Package management system with the following features:

(a) A packages.json in ZK to store the configuration,
(b) APIs to read/edit them, and
(c) Isolated classloaders to load the classes from those packages when the
'class'
attribute is prefixed with '<package-name>:'
(noble, Ishan Chattopadhyaya)

* SOLR-13662: A CLI based Package Manager ("bin/solr package help" for more
details).
(Ishan Chattopadhyaya, noble, David Smiley, Jan Hoydahl)

* SOLR-10786: Add DBSCAN clustering Streaming Evaluator (Joel Bernstein)

* SOLR-13911: Add 'missing' aggregation in JSON FacetModule (hossman, Munendra S N)

* SOLR-13912: Add 'countvals' aggregation in JSON FacetModule (hossman, Munendra S
N)

* SOLR-12217: Support shards.preference in SolrJ for single shard collections. The

parameter
is now used by the CloudSolrClient and Streaming Expressions. (Houston Putman,
Tomas Fernandez-Lobbe)

* SOLR-14043: Allow the precision Stream Evaluator to operate on matrices (Joel

bernstein)

* SOLR-14060: Allow the random Streaming Expression to create the x-axis (Joel
Bernstein)

Improvements
---------------------

* SOLR-13731: 'javabin' must support a 1:1 mapping of the JSON update format
(noble)

* SOLR-13831: Support defining arbitrary autoscaling simulation scenarios. (ab)

* SOLR-13865: Move replica routing code to SolrJ. (Houston Putman via Tomas
Fernandez-Lobbe)

* SOLR-13844: Remove replica recovery terms with the replica term (Houston Putman
via Cao Manh Dat)

* SOLR-13898: Use the new SolrCache.computeIfAbsent instead of non-atomic get / put

in order to
improve cache hit ratios. (ab, Ben Manes)

* SOLR-13907: In the Cloud/tree section of the Admin UI, fix metadata panel and
force an horizontal scrollbar for the tree
(Richard Goodman via Tomás Fernández Löbbe)

* SOLR-13905: Make findRequestType in AuditEvent more robust (janhoy)

* SOLR-13961: When using partial/atomic updates to remove child documents, we now

support setting to null or
an empty list as equivalent to the "remove" command. (Thomas Wöckinger)

* SOLR-13969: Clean up and document AuditEvent API (janhoy)

* SOLR-13968: Support postingsFormat and docValuesFormat in schema fields. (Bruno

Roustant)

* SOLR-11706: Add support for aggregation on multivalued fields in JSON facets.

min, max, avg, sum, sumsq, stddev,
variance, percentile aggregations now have support for multivalued fields.
(hossman, Munendra S N)

* SOLR-13987: Admin UI should not rely on javascript eval() (rmuir, Kevin Risden)

* SOLR-13979: Expose separate metrics for distributed and non-distributed requests.

(ab)

* SOLR-13957: Add sensible defaults for the facet, random, facet2D, timeseries,
stats
 and update Streaming Expressions (Joel Bernstein)

* SOLR-13904: Analytic component abandon requests exceedig limit passed via

timeAllowed parameter
(Houston Putman, Mikhail Khludnev).

* SOLR-13970: Fail the request when collapsing or expand is used with Grouping.
(Erick Erickson, Joel Bernstein, Munendra S N)

Bug Fixes
---------------------

* SOLR-13827: Fail on unknown operation in Request Parameters API (Munendra S N,

noble)

* SOLR-13403: Fix NPE in TermsComponent for DatePointField (yonik, Munendra S N)

* SOLR-9802: Fix grouping failure for date field in solrcloud (Erick Erickson,
Munendra S N, Vitaly Lavrov)

* SOLR-12393: Compute score if requested even when expanded docs not sorted by
score in ExpandComponent.
(David Smiley, Munendra S N)

* SOLR-13877, SOLR-7798: Robust support for ExpandComponent when used independently

of CollapsingPostFilter. (Jörg Rathlev, Michael Gibney, Munendra S N)

* SOLR-13823: Fix ClassCastEx when score is requested with group.query. This also
fixes score not being generated
for distributed group.query case. (Uwe Jäger, Munendra S N)

* SOLR-13207: In dismax and edismax "minimum should match" queries, the < operator
not followed by a number will now return
an HTTP 400 error rather than 500, and the error message will explain that < must
be followed by a number. (Chris Hennick)

* SOLR-13762: Fix BinaryField support for non-binary wt's (Thomas Woeckinger via
Jason Gerlowski)

* SOLR-13882: Collections API COLSTATUS does not check live_nodes when reporting
replica's status.
(Erick Erickson, Andrzej Białecki)

* SOLR-13872: Fixed Backup failures - typically manifesting as NoSuchFileException

- due to race conditions
in saving/reserving commit points (hossman)

* SOLR-13921: Processing UpdateRequest with delegation token throws

NullPointerException (Istvan Farkas via Kevin Risden)

* SOLR-13941: Tests configure Jetty differently than when running via start.jar
(janhoy, Uwe Schindler)

* SOLR-13950: Fix InterruptedException being swallowed in ZkStateReader’s

getLeaderRetry method
(Andy Vuong via Tomás Fernández Löbbe)

* SOLR-13465: CoreContainer.auditloggerPlugin should be volatile (janhoy, hossman)

* SOLR-13805: NPE when calling /solr/admin/info/health on standalone solr (Nicholas
DiPiazza, shalin)

* SOLR-13954: Embedded ZooKeeper in Solr now does not try to load JettyAdminServer
(janhoy)

* SOLR-13087: Use EUID instead of whoami to check for root in the bin/solr script
(Martijn Koster via Jason Gerlowski)

* SOLR-13977: solr create -c not working under Windows 10 (janhoy)

* SOLR-13966: LatLonPointSpatialField wasn't working correctly with atomic/partial

updates or RTG.
(Thomas Wöckinger via David Smiley)

* SOLR-13563: SPLITSHARD using LINK method fails on disk usage checks. (Andrew
Kettmann, ab)

* SOLR-13806: SolrJ QueryResponse._explainMap is incorrectly typed. (Guna Sekhar

Dorai, ab)

* SOLR-13975, SOLR-13896: ConcurrentUpdateSolrClient connection stall prevention.

(ab, caomanhdat)

* SOLR-13953: Prometheus exporter in SolrCloud mode limited to 100 nodes (Alex

Jablonski via Erick Erickson)

* SOLR-13945: Fix: SPLITSHARD can cause data loss on a failure to commit after the
sub-shards are active and a rollback
is done to make parent shard active again (Ishan Chattopadhyaya, ab)

* SOLR-14071: Untrusted configsets cannot use <lib> directive due to security

reasons (Ishan Chattopadhyaya)

* SOLR-14013: FIX: javabin performance regressions (noble, yonik, Houston Putman)

* SOLR-14079: SPLITSHARD splitByPrefix doesn't work in async mode. This also

affects splits triggered by the autoscale framework, which use async mode.
(Megan Carey, Andy Vuong, Bilal Waheed, Ilan Ginzburg, yonik)

Other Changes
---------------------

* SOLR-12769: Fix incorrect documentation for 'delete' op in Request parameters API

(Alexandre Rafalovitch, Munendra S N)

* SOLR-13841: Provide mappings for jackson annotation @JsonProperty to use Jackson

deserializer (noble)

* SOLR-13824: Strictly reject anything after JSON in most APIs (Mikhail Khludnev,
Munendra S N)

* SOLR-13869: SolrTestCaseJ4.getNextAvailablePort() has been deprecated (and will

be removed in 9.0) since
it is fundementally flawed. Users with tests that depend on this method should
restructure them. (hossman)

* SOLR-13782: Solr Ref Guide no longer published in PDF format. (Cassandra Targett)
* SOLR-12193: Move some log messages to TRACE level (gezapeti, janhoy)

* SOLR-13885: Typos in the documentation. (KoenDG via Cassandra Targett)

* SOLR-13992: Code refactored to have collection name, slice name in Replica, Slice
(noble)

* SOLR-13998: Add thread safety annotations to Solr. This only introduces the
annotations and doesn't add these to
existing classes. (Anshum Gupta, Mark Miller)

* SOLR-13818: Upgrade jackson to 2.10.0 (marungo, Andras Salamon via Kevin Risden)

* SOLR-13999: Document zkHost param on topic() stream (Vergil Cheynov via Jason
Gerlowski)

================== 8.3.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.5.5
Jetty 9.4.24.v20191120

Upgrade Notes
----------------------

* Velocity: The initialization parameters

"params.resource.loader.enabled" and "solr.resource.loader.enabled" have been
removed and replaced by
Java system properties "velocity.resourceloader.params.enabled" and
"velocity.resourceloader.solr.enabled"
respectively (both turned off by default). Please see SOLR-13971 for more
details.

Bug Fixes
----------------------

* SOLR-13963: JavaBinCodec has concurrent modification of CharArr resulting in

corrupt internode updates (Colvin Cowie, noble)

* SOLR-13971: Velocity response writer's resource loading now possible only through
startup parameters.
(Ishan Chattopadhyaya, David Smiley)

* SOLR-13465: CoreContainer.auditloggerPlugin should be volatile (janhoy, hossman)

* SOLR-13905: Make findRequestType in AuditEvent more robust (janhoy)

* SOLR-13941: Tests configure Jetty differently than when running via start.jar
(janhoy, Uwe Schindler)

================== 8.3.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.5.5
Jetty 9.4.19.v20190610

Upgrade Notes
----------------------

* Users who have written test cases that extend SolrTestCaseJ4 may see
NullPointerExceptions if
their tests directly reference both SolrTestCaseJ4.initCoreDataDir and
SolrTestCaseJ4.deleteCore().
This change in behavior is due to a bug fix in deleteCore() to ensure the dataDir
is properly reset
in tests that call initCore()/deleteCore() multiple times in a given test
(class). initCoreDataDir
is now deprecated, and users are encouraged to use
SolrTestCaseJ4.initAndGetDataDir() in it's place.
See SOLR-13664 for more details.

* For JWTAuthPlugin, the 'jwkUrl' configuration key is deprecated and may be

removed later, please use 'jwksUrl'
instead. See SOLR-13734.

New Features
----------------------

* SOLR-13375: Two dimensional routed aliases are now available for organizing
collections based on the data values
of two fields. (Gus Heck)

* SOLR-13399: SPLITSHARD implements a new splitByPrefix option that takes into

account the actual document distribution
when using compositeIds. Document distribution is calculated using the
"id_prefix" field (if it exists) containing
just the compositeId prefixes, or directly from the indexed "id" field otherwise.
(yonik, Megan Carey)

* SOLR-13622: Add cat() stream source to create tuples from lines in local files
(Jason Gerlowski and Joel Bernstein)

* SOLR-11866: QueryElevationComponent can have query rules configured with

match="subset" wherein the words need only
match a subset of the query's words and in any order. (Bruno Roustant via David
Smiley)

* SOLR-13682: command line option to export documents to a file (noble)

* SOLR-13257: Support deterministic replica routing preferences for better cache

usage (Michael Gibney
via Christine Poerschke, Tomás Fernández Löbbe)

* SOLR-13122: Ability to query aliases in Solr Admin UI (janhoy)

* SOLR-13713: JWTAuthPlugin to support multiple JWKS endpoints (janhoy)

* SOLR-13734: JWTAuthPlugin now supports multiple IdP issuers through configuring a

new 'issuers' configuration key.
Access tokens issued and signed by any of the configured issuers will be
validated (janhoy)

* SOLR-13272: Add support for arbitrary ranges in JSON facet's Range facets.
(Apoorv Bhawsar, Munendra S N, Mikhail Khludnev, Ishan Chattopadhyaya, Jan
Høydahl)

* SOLR-13632: Support integral plots, cosine distance and string truncation with
math expressions (Joel Bernstein)

* SOLR-13667: Add upper, lower, trim and split Stream Evaluators (Joel Bernstein)

* SOLR-13625: Add CsvStream, TsvStream Streaming Expressions and supporting Stream

Evaluators (Joel bernstein)

* SOLR-8241: Add CaffeineCache, an efficient implementation of SolrCache.(Ben

Manes, Shawn Heisey, David Smiley, Andrzej Bialecki)

* SOLR-13298: Allow zplot to plot matrices (Joel Bernstein)

Improvements
----------------------

* SOLR-12368: Support InPlace DV updates for a field that does not yet exist in any
documents
(hossman, Simon Willnauer, Adrien Grand, Munendra S N)

* SOLR-13558, SOLR-13693: Allow dynamic resizing of SolrCache-s. (ab)

* SOLR-6305: Ability to set the replication factor for index files created by
HDFSDirectoryFactory (Boris Pasko via Kevin Risden)

* SOLR-13702: Some components register twice their metric names (janhoy)

* SOLR-11601: Improved error message when geodist(llpsf) is used with arguments

referring to a LatLonPointSpatialField.
(Amrit Sarkar)

* SOLR-13542: Code cleanup - Avoid using stream filter count where possible (Koen
De Groote via Tomás Fernández Löbbe)

* SOLR-13720: BlockJoinParentQParser.getCachedFilter()made public for accessing

from QParser plugins
(Stanislav Livotov via Mikhail Khludnev)

* SOLR-13728: If a partial update (aka atomic update) is attempted on a document

that has child docs, then ensure
the schema supports it (_root_ stored/docValues) by throwing an exception. (David
Smiley)

* SOLR-13742: Allow optional redaction of data saved by 'bin/solr autoscaling -

save'. (ab)

* SOLR-13739: Optimized large managed schema modifications; Internal O(n^2)

problem. (Thomas Wöckinger via David Smiley)

* SOLR-9658: Max idle time support for SolrCache implementations. (hoss, ab)

* SOLR-13763: Improve the tracking of "freedisk" in autoscaling simulations. (ab)

* SOLR-13773: Add Prometheus Exporter GC and Heap options. (Houston Putman via
Anshum Gupta, David Smiley)

* SOLR-13638: Add debug, trace logging to RuleBasedAuthorizationPlugin (Jason

Gerlowski)

* SOLR-13784: EmbeddedSolrServer's defaultCore constructor argument is now optional

(David Smiley)

* SOLR-13798: SSL: Adding Enabling/Disabling client's hostname verification config

(Cao Manh Dat)

* SOLR-13771: Add -v and -m to ulimit section of reference guide and bin/solr

checks (Erick Erickson)

* SOLR-13795: Managed schema operations should do a core reload in Solr standalone

mode.
(Thomas Wöckinger via David Smiley)

* SOLR-13719: Introducing SolrClient.ping(collection) in SolrJ (Geza Nagy via

Mikhail Khludnev)

Bug Fixes
----------------------

* SOLR-13206: Fix AIOOBE when group.facet is specified with group.query and return
proper error code. (Marek, Munendra S N)

* SOLR-11556: Backup and restore command really supports picking one of a few
repositories by
repository parameter (Timothy Potter via Mikhail Khludnev)

* SOLR-13647: Default solr.in.sh contains incorrect default value (John Ryan)

* SOLR-13660: Fixed AbstractFullDistribZkTestBase.waitForActiveReplicaCount() to

ensure
replicas are active. (hossman)

* SOLR-13664: Fixed SolrTestCaseJ4.deleteCore() to properly reset the dataDir used

by initCore()
(hossman)

* SOLR-13679: Default style of ExplainDocTransformer registered via solrconfig.xml

should be same as default style
of ExplainDocTransformer registered in defaultFactories (Munendra S N)

* SOLR-13672: Cloud -> Zk Status page now parses response from Zookeeper 3.5.5
correctly (Jörn Franke, janhoy, Shawn Heisey)

* SOLR-13674: NodeAddedTrigger does not support configuration of replica type hint.

A new replicaType property
has been added to NodeAddTrigger so that new replicas of the given type are
added. The default value is `NRT`.
 (Irena Shaigorodsky via shalin)

* SOLR-13141: CDCR bootstrap does not replicate index to the replicas of target
cluster.
(Krzysztof Watral, Amrit Sarkar, Tim, Tdspringsteen, shalin)

* SOLR-13694: IndexSizeEstimator NullPointerException. (ab, hossman)

* SOLR-13700: Fixed a race condition when initializing metrics for new security
plugins on security.json change (hossman)

* SOLR-13701: Fixed JWTAuthPlugin to update metrics prior to continuing w/other

filters or returning error (hossman)

* SOLR-6328: facet.missing=true should return missing count even when facet.limit=0

(hossman, Munendra S N)

* SOLR-13704: Improve error message and change error code to 400 for client errors
in ExpandComponent (Munendra S N)

* SOLR-13699 - maxChars no longer working on CopyField with javabin (Chris Troullis

via noble)

* SOLR-13718: SPLITSHARD (async) with failures in underlying sub-operations can

result in data loss (Ishan Chattopadhyaya)

* SOLR-13717: Fixed distributed grouping when multiple 'fl' params are specified
(hossman, Christine Poerschke)

* SOLR-13240: Fixed UTILIZENODE action resulting in IllegalArgumentException.

(Hendrik Haddorp, Richard Goodman, Tim Owen, shalin, noble, Christine Poerschke)

* SOLR-13238: BlobHandler generates non-padded md5 (Jeff Walraven via janhoy)

* SOLR-13780: Fix ClassCastException in NestableJsonFacet (Tiago Martinho de

Barros, Munendra S N)

* SOLR-13725: Allow negative values for limit in TermsFacetMap (Richard Walker,

Munendra S N)

* SOLR-13022: Fix NPE when sorting by non-existent aggregate function in JSON Facet
(hossman, Munendra S N)

* SOLR-13727: Fixed V2Requests - HttpSolrClient replaced first instance of "/solr"

with "/api" which
caused a change in host names starting with "solr". (Megan Carey via yonik)

* SOLR-13180: Fix ClassCastException in Json Request API (Johannes Kloos, Jan

Høydahl, Munendra S N)

* SOLR-13417: Handle stats aggregation on date and string fields in SolrJ's JSON
facet response processing
(Jason Gerlowski, Munendra S N)

* SOLR-13712: JMX MBeans are not exposed because of race condition between creating
platform mbean server and
registering mbeans. (shalin)

* SOLR-13802: Managed schema manipulations were not persisting the optional

luceneMatchVersion that can be set
on an Analyzer. (Thomas Wöckinger)

* SOLR-13539: Fix for class-cast issues during atomic-update 'removeregex'

operations. This also incorporated some
tests Tim wrote as a part of SOLR-9505. (Tim Owen via Jason Gerlowski)

* SOLR-13790: LRUStatsCache size explosion and ineffective caching. (ab)

* SOLR-13376: Multi-node race condition to create/remove nodeLost markers. (hoss,

ab)

* SOLR-13293: ConcurrentUpdateHttp2SolrClient always log AsynchronousCloseException

exception error on indexing.
(Cao Manh Dat)

* SOLR-13828: Improve ExecutePlanAction error handling. (ab)

* SOLR-13760: Fix regression in support for Date math in TRA start date that was
introduced by SOLR-13375

* SOLR-13829: RecursiveEvaluator casts Continuous numbers to Discrete Numbers,

causing mismatch (Trey Grainger, Joel Bernstein)

* SOLR-13815: Live shard split (where updates actively continue during the split)
can lose updates due to cluster
state happening to change between checking if the current shard is active and
later checking if there are any
sub-shard leaders to forward the update to. (yonik)

* SOLR-13665: Added missing netty dependencies to solrJ and upgraded netty to v

4.1.29.Final (Jörn Franke, janhoy)

* SOLR-13793: HttpSolrCall now maintains internal request count (_forwardedCount)

for remote queries and limits them to
the number of replicas. This avoids making too many cascading calls to remote
servers, which, if not restricted, can
bring down nodes containing the said collection (Kesharee Nandan Vishwakarma,
Ishan Chattopadhyaya)

* SOLR-13834: ZkController#getSolrCloudManager() created a new instance of

ZkStateReader, thereby causing mismatch in the
visibility of the cluster state and, as a result, undesired race conditions (Clay
Goddard via Ishan Chattopadhyaya)

* SOLR-13835: HttpSolrCall produces incorrect extra AuditEvent on

AuthorizationResponse.PROMPT (janhoy, hossman)

* SOLR-13843: The MOVEREPLICA API ignores replica type and always adds 'nrt'
replicas (Amrit Sarkar via shalin)

* SOLR-13677: All Metrics Gauges should be unregistered by components that

registered them. (noble, ab)

* SOLR-13855: DistributedZkUpdateProcessor should have been propagating

URP.finish() lifecycle like it used to before
8.1 (a regression). Impacts integrity since Run URP's finish() propagates this
to the updateLog to fsync.
(David Smiley)
Other Changes
----------------------

* SOLR-13779: Use the safe fork of simple-xml for clustering contrib. (Dawid Weiss)

* SOLR-13585: Factor out SearchGroupsResultTransformer.[de]serializeOneSearchGroup

methods. (Christine Poerschke, Diego Ceccarelli)

* SOLR-12870: Use StandardCharsets instead of String values (Peter Somogyi via

Munendra S N)

* SOLR-10377: Add `debug.explain.structured` to Admin UI. (David Smiley, Munendra S

N)

* SOLR-13629: Clean trailing whitespace from 'analytics' contrib (Neal Sidhwaney

via Jason Gerlowski)

* SOLR-13643: Add Getters/Setters in ResponseBuilder for analytics response

handling (Neal Sidhwaney via Munendra S N)

* SOLR-13680: Use try-with-resource to close the closeable resource (Furkan KAMACI,

Munendra S N)

* SOLR-13573: Add SolrRangeQuery getters for upper, lower bound (Brian Rhees via
Jason Gerlowski)

* SOLR-13658: Precommit fail Java "var" until 9x. Fail "var...<>" constructs
entirely (Erick Erickson)

* SOLR-13767: Upgrade jackson to 2.9.9 (janhoy)

* SOLR-11492: Clean up /solr/cloud-dev scripts and provide a single well documented

script (Gus Heck, Robert Bunch)

* SOLR-13747: New
TestSSLTestConfig.testFailIfUserRunsTestsWithJVMThatHasKnownSSLBugs() to give
people running
tests more visibility if/when they use a known-buggy JVM causing most SSL tests
to silently SKIP. (hossman)

* SOLR-13791: Remove remaining Commons BeanUtils references. (Andras Salamon,

Christine Poerschke)

* SOLR-13812: Add javadocs, uneven rejection and basic test coverage for the
SolrTestCaseJ4.params method.
(Diego Ceccarelli, Christine Poerschke, Munendra S N)

* SOLR-13787: An annotation based system to write v2 APIs (noble)

* SOLR-12786: Update Ref Guide build tool versions (Cassandra)

================== 8.2.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.5.5
Jetty 9.4.19.v20190610

Upgrade Notes
----------------------

* SOLR-8346: Upgrade ZooKeeper to 3.5.5. ZooKeeper 3.5.5 introduces more security

features. Include in your zoo.cfg
file at minimum the following:
4lw.commands.whitelist=mntr,conf,ruok
You can use
4lw.commands.whitelist=* to enable all ZooKeeper "4 letter commands".
(Erick Erickson)

* SOLR-13515: org.apache.solr.util.SolrPluginUtils.IdentityRegenerator has been

removed, please use
the identical org.apache.solr.search.NoOpRegenerator instead. (David Smiley,
Christine Poerschke)

* SOLR-13541: Upgrade Jetty to 9.4.19.v20190610. (Erick Erickson, Cao Manh Dat)

* SOLR-13420: Routed Aliases now use collection properties rather than core
properties to identify collections that
belong to the alias by default. This should be invisible and fully backwards
compatible from within solr, and
existing routed alias collections with core based properties will continue to
work, but new collections created will
not add a property to core.properties anymoore so any external code that
inspected core.properties will not find the
'routedAliasName' key in new cores belonging to routed aliases.

* SOLR-13419: Time Routed Aliases now have a '__TRA__' infix preceding the
timestamp portion of the collection names.
collections with the old format will not be altered and will continue to work,
but any new TRA's or new collections
for existing TRA's will use the new format. Solr will handle this invisibly, but
any external code that attempted to
predict collection names in TRA's will probably need adjustment.

* SOLR-13507: Support for "addr" parameter from the "/solr/admin/zookeeper"

endpoint has now been removed and will no
longer be supported.

New Features
----------------------

* SOLR-13320: add an update param failOnVersionConflicts=false to updates not fail

when there is a version conflict (noble)

* SOLR-13445: Preferred replicas on nodes with same system properties as the query
master (Cao Manh Dat)

* SOLR-13047: Add facet2D Streaming Expression (Nazerke Seidan, Joel Bernstein)

* SOLR-13494: Add DeepRandomStream implementation (Joel Bernstein)

* SOLR-13434: OpenTracing support for Solr (Cao Manh Dat)

* SOLR-13512: Raw index data analysis tool (extension of COLSTATUS collection

command). (ab)

* SOLR-13550: Allow zplot to automatically create the x axis (Joel Bernstein)

* SOLR-13552: Add recNum Stream Evaluator (Joel Bernstein)

* SOLR-13560: Add isNull and notNull Stream Evaluators (Joel Bernstein)

* SOLR-10291: Add matches Stream Evaluator to support regex matching (Joel

Bernstein)

* SOLR-13589: Allow zplot to visualize 2D clusters and convex hulls (Joel

Bernstein)

Improvements
----------------------

* SOLR-13049: Make contrib/ltr Feature.defaultValue configurable. (Stanislav

Livotov, Christine Poerschke)

* SOLR-13306: Add a request parameter to execute a streaming expression locally

* SOLR-13440: Support saving/restoring autoscaling state for repeatable

simulations. (ab)

* SOLR-11558: It would be nice if the Graph section of the Cloud tab in the Admin
UI could give some more
information about the replicas of a collection (Erick Erickson)

* SOLR-13468: autoscaling/suggestions should be able to give suggestions from

config sent as a payload (noble)

* SOLR-12304: The MoreLikeThisComponent now supports the mlt.interestingTerms

parameter. Previously this option was
unique to the MLT handler. (Alessandro Benedetti via David Smiley)

* SOLR-13484: autoscaling/diagnostics API should be able to give diagnostics output

from config sent as a payload (noble)

* SOLR-13493: /autoscaling/suggestions to be able to filter by type (noble)

* SOLR-13504: In autoscaling policies, use an explicit 'nodeset' attribute for

filtering
nodes instead of using them directly at the toplevel (noble)

* SOLR-13329: In autoscaling policies, use an explicit 'put : on-each-node'

to specify the the rules is applied on each node (noble)

* SOLR-13347: Transaction log to natively support UUID types (Thomas Wöckinger via
noble)

* SOLR-13367: Highlighting: Range queries will now highlight in hl.method=unified

mode. (David Smiley)

* SOLR-13569: Admin UI visual indication of prod/test/dev environment (janhoy)

* SOLR-12554: Expose IndexWriterConfig's ramPerThreadHardLimitMB through
solrconfig.xml (Ishan Chattopadhyaya, Munendra S N)

* SOLR-13602 SOLR-13588: Add a field type for Estonian language to default

managed_schema,
document about Estonian language analysis in Solr Ref Guide (Tomoko Uchida)

* SOLR-13003: Query Result Cache does not honor maxRamBytes parameter. (ab, Brian
Ecker)

Bug Fixes
----------------------

* SOLR-13474: Fix "Search is temporarily disabled" logic to be consistent for

entire request (hossman)

* SOLR-13469: Fix rejected requests during full disk full + IndexFetch to use 503
not 403 (hossman)

* SOLR-12941: Fix IndexSizeTrigger to correctly work with "aboveBytes" and

"splitMethod=link" parameters. (ab)

* SOLR-13491: SolrZkClient's watch wrapper no longer allows zookeeper to hold the

same watch object multiple times.

* SOLR-13489: Stop the leader from trying to rejoin the election on session
expiration and harden our zk reconnect code path.
(Mark Miller, Anshum Gupta)

* SOLR-12249: Better error message when grouping on a tokenized (non SortableText)

field in SolrCloud (Erick Erickson)

* SOLR-13521: Fix input parameter handling in SchemaRequest.DynamicField and

SchemaRequest.FieldTypes (Schema API)
(Tomás Fernández Löbbe)

* SOLR-13496: Fix distributed grouping related NullPointerException in

JSONWriter.writeSolrDocument (Christine Poerschke)

* SOLR-12013: collections API CUSTERSTATUS command fails when configset missing

(Erick Erickson)

* SOLR-13509: NPE on omitHeader=true is fixed by sending omitHeader=false to shard

searches (Munendra S N, Mikhail Khludnev)

* SOLR-13333: unleashing terms.ttf from terms.list when distrib=false (Munendra S N

via Mikhail Khludnev)

* SOLR-13490: Fix CollectionStateWatcher/CollectionStatePredicate based APIs in

ZkStateReader and
CloudSolrClient to be triggered on liveNode changes. Also add
Predicate<DocCollection> equivalents
for callers that don't care about liveNodes. (hossman)

* SOLR-13545: ContentStreamUpdateRequest refused to close file (Colvin Cowie,

Mikhail Khludnev)

* SOLR-12127: Updates containing 'set' operation with value null or empty list
should be considered as Atomic Update
 (Oliver Kuldmäe, Munendra S N, Ishan Chattopadhyaya)

* SOLR-13187: Fix NPE when invalid query parser is specified and return 400 error
code.
(Cesar Rodriguez, Marek, Charles Sanders, Munendra S N, Mikhail Khludnev)

* SOLR-12979: Improve error message and change error code to 400 when collapse
field is non-docValued and
non-uninvertible (hossman, Munendra S N)

* SOLR-13574: Fix many test and test-framework classes to not fail on

After/AfterClass cleanup if
assumptions fail in Before/BeforeClass setup (hossman)

* SOLR-13280: Strengthen ScheduledTrigger's preferredOperation parameter

validation. (Christine Poerschke)

* SOLR-9409: Improve error message for unsupported field types and return 400 error
code on
unsupported values in collapsing (hossman, Munendra S N)

* SOLR-13404: Support group.query in multi-shard environment when group.main=true

or group.format=simple (Munendra S N)

* SOLR-13538: toNativeType () TrieDate & EnumField do not handle CharSequence

properly (Munendra S N)

* SOLR-13532: Fix http timeout and error logging bugs in RecoveryStrategy. (Suril
Shah, hossman)

* SOLR-13472: Forwarded requests should skip authorization on receiving nodes

(adfel, Ishan Chattopadhyaya)

* SOLR-13619: Kerberos plugin to pass original user principal to avoid 403 on nodes
not hosting a collection
(adfel, Ishan Chattopadhyaya, noble)

Other Changes
----------------------

* SOLR-13453: Adjust auth metrics asserts in tests caused by SOLR-13449 (janhoy)

* SOLR-13437: noggit json parser is forked into solrj (noble)

* SOLR-13467: Include the S2 Geometry lib to make it simpler to use prefixTree="s2"

on a Geo3D spatial field.
Improved documentation on Geo3D too. (David Smiley)

* SOLR-13371: Re-structure and clarify Security chapters in Ref Guide (janhoy)

* SOLR-13235: Split Collections API Ref Guide page into several smaller child pages
(Cassandra Targett)

* SOLR-8754: Adding test cases for org.apache.solr.util.hll.NumberUtilTest (Benoit

Vanalderweireldt via janhoy)

* SOLR-13511: Add SearchHandler.newResponseBuilder method to facilitate custom

plugins' maintenance
of per-request state in a custom ResponseBuilder. (Ramsey Haddad, Christine
Poerschke)

* SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException

when tests/seeds
request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs
(hossman, Cao Manh Dat)

* SOLR-13279: Clarify ScheduledTrigger's "every parameter missing" error response.

(Christine Poerschke)

* SOLR-13576: Factor out a TopGroupsShardResponseProcessor.fillResultIds method.

(Christine Poerschke, Diego Ceccarelli)

* SOLR-12364: Add test cases for edismax boost (David Smiley, Munendra S N)

* SOLR-13603: Remove usage of GroupingSpecification's deprecated methods (Munendra

S N)

* SOLR-13507: Remove support for "addr" parameter from the "/solr/admin/zookeeper"

endpoint. (janhoy, Anshum Gupta)

* SOLR-13588: Document Estonian analyzer in Solr Ref Guide (Tomoko Uchida)

* SOLR-13602: Add a field type for Estonian language to default managed_schema

(Tomoko Uchida)

================== 8.1.2 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.4.14
Jetty 9.4.14.v20181114

Bug Fixes
----------------------

* SOLR-13510: Intermittent 401's for internode requests with basicauth enabled (Cao
Manh Dat, Colvin Cowie)

* SOLR-13523: In 8.1, Atomic Updates were broken (NPE) when the schema declared the
new _nest_path_ field even if you
weren't using nested docs. In-place updates were not affected (worked).
Reminder: if you don't need nested docs then remove both _root_ and _nest_path_ !
(David Smiley)

* SOLR-13566: REINDEXCOLLECTION does not work with (basic) authentication. (Colvin

Cowie, ab)

* SOLR-13159: Fix atomic update encoding issue for UUID, enum, bool, and binary
fields (Thomas Wockinger via Jason Gerlowski)

* SOLR-13583: Impossible to delete a collection with the same name as an existing

alias. This fixes also a bug in
 REINDEXCOLLECTION when used with removeSource=true which could lead to a data
loss. (ab)

Improvements
----------------------

* SOLR-13158: DataImportHandler: Added enable.dih.dataConfigParam system property

to toggle whether the dataConfig param
is permitted. (David Smiley, janhoy, Tomás Fernández Löbbe)

================== 8.1.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.4.14
Jetty 9.4.14.v20181114

Bug Fixes
----------------------

* SOLR-13475: Null Pointer Exception when querying collection through collection

alias. (Jörn Franke, ab)

================== 8.1.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 2.0 and Velocity Tools 3.0
Apache ZooKeeper 3.4.14
Jetty 9.4.14.v20181114

Upgrade Notes
----------------------

* Solr's default behavior when dealing with 'maxBooleanClauses' has changed to

reduce the risk of exponential
query expansion when dealing with pathological query strings. A default upper
limit of 1024 clauses
(The same default prior to Solr 7.0) is now enforced at the node level, and can
be overridden in solr.xml.
The identically named solrconfig.xml setting is still available for limiting the
size of 'explicit' boolean
query strings, but this per-collection limit is still ristricted by the upper-
bound of the global limit
in solr.xml. See SOLR-13336 for more details.

* When requesting the status of an async request via REQUESTSTATUS collections API,
the response will
 include the list of internal async requests (if any) in the "success" or "failed"
keys (in addition
to them being included outside those keys for backwards compatibility). See SOLR-
12708 for more
details

* SOLR-13335: Velocity and Velocity Tools were both upgraded as part of this
release. Velocity upgraded from 1.7 to 2.0.
Please see https://velocity.apache.org/engine/2.0/upgrading.html about upgrading.
Velocity Tools upgraded from 2.0 to
3.0. For more details, please see
https://velocity.apache.org/tools/3.0/upgrading.html for details about the upgrade.

* SOLR-13407: Update requests sent to non-routed aliases that point to multiple

collections are no longer accepted. Until
now Solr followed an obscure convention of updating only the first collection
from the list, which usually was not what
the user intended. This change explicitly rejects such update requests.

* SolrGangliaReporter has been removed from Solr because support for Ganglia has
been removed from Dropwizard Metrics 4
due to a transitive dependency on LGPL.

* Custom TransientSolrCoreCache implementations no longer use the

Observer/Observable pattern. To notify Solr that
a core has been aged out of the cache, call
CoreContainer.queueCoreToClose(SolrCore). See SOLR-13400 for details.

* SOLR-13394: The default GC has been changed from CMS to G1. To override this (in
order to switch to CMS or any
other GC), use GC_TUNE section of bin/solr.in.sh or bin/solr.in.cmd.

* SOLR-5970: Until now, the CREATE command of Collections API returned status 0
even in case of failure. Now, the status
code will be non-zero in case of failures (e.g. 4xx, 5xx).

New Features
----------------------
* SOLR-13131: Category Routed Aliases are now available for data driven assignment
of documents to collections based on
values of a field. The Ref Guide now has a page dedicated to explaining the
different types of aliases. (Gus Heck,
Moshe Bla)

* SOLR-13171 : A true streaming parser for javabin payload/stream without creating

any objects (noble)

* SOLR-13261: Make SortableTextField work with export/streaming. NOTE: requires

that the field have
useDocValuesAsStored=true (either explicit or as the default).

* SOLR-12121: JWT Token authentication plugin with OpenID Connect implicit flow
login through Admin UI (janhoy)

* SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins

(janhoy)

* SOLR-10436: Add hashRollup Streaming Expression (Joel Bernstein)

* SOLR-13276: Adding Http2 equivalent classes of CloudSolrClient and
HttpClusterStateProvider (Cao Manh Dat)

* SOLR-13287: Allow zplot to visualize probability distributions in Apache Zeppelin

(Joel Bernstein)

* SOLR-13271: Read-only mode for SolrCloud collections (ab, shalin)

* SOLR-13292: Provide extended per-segment status of a collection. (ab)

* SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections.

This issue also adds
a back-compat check of the .system collection to notify users of potential
compatibility issues after
upgrades or schema changes. (ab)

* SOLR-13374: Add fetchSize parameter to the jdbc Streaming Expression (Joel

Bernstein)

* SOLR-12638: Partial/Atomic Updates for nested documents. This enables atomic

updates for nested documents, without
the need to supply the whole nested hierarchy (which would be overwritten if
absent). This is done by fetching the
whole document hierarchy, updating the specific doc in the path that is to be
updated, removing the old document
hierarchy and indexing the new one with the atomic update merged into it. Also,
[child] Doc Transformer now works
with RealTimeGet. (Moshe Bla, David Smiley)

* SOLR-13262: Add collection RENAME command and support using aliases in most
collection admin commands. (ab)

* SOLR-13391: Add variance and standard deviation stream evaluators (Nazerke

Seidan, Joel Bernstein)

* SOLR-13427: Support simulating the execution of autoscaling suggestions. (ab)

Bug Fixes
----------------------

* SOLR-12330: 500 error code on json.facet syntax errors (Munendra S N, Mikhail

Khludnev)

* SOLR-13229: Cleanup replicasMetTragicEvent after all types of exception (Tomás

Fernández Löbbe)

* SOLR-11876: In-place update fails when resolving from Tlog if schema has a
required field (Justin Deoliveira, janhoy,
Ishan Chattopadhyaya)

* SOLR-12708: Async collection actions should not hide internal failures (Mano
Kovacs, Varun Thacker, Tomás Fernández Löbbe)

* SOLR-11883: 500 code on functional query syntax errors and parameter

dereferencing errors
(Munendra S N via Mikhail Khludnev)

* SOLR-13234: Prometheus Metric Exporter not threadsafe. This changes the

prometheus exporter to collect metrics
 from Solr on a fixed interval controlled by this tool and prevents concurrent
collections. This change also improves
performance slightly by using the cluster state instead of sending multiple HTTP
requests to each node to lookup
all the cores.
(Danyal Prout via shalin)

* SOLR-9882: 500 error code on breaching timeAllowed by core and distributed (fsv)
search,
old and json facets (Mikhail Khludnev)

* SOLR-13285: Updates with enum fields and javabin cause ClassCastException (noble)

* SOLR-13295: Reproducible failure in TestDistributedGrouping (Erick Erickson)

* SOLR-13254: Correct message that is logged in solrj's ConnectionManager when an

exception
occurred while reconnecting to ZooKeeper. (hu xiaodong via Christine Poerschke)

* SOLR-13284: NullPointerException with 500 http status on omitted or wrong wt

param.
It's fixed by fallback to json (Munendra S N via Mikhail Khludnev)

* SOLR-13244: Admin UI Nodes view fails and is empty when a node is temporarily
down (janhoy)

* SOLR-13253: IndexSchema.getResourceLoader was being used to load non-schema

things, which can be a memory leak if
"shareSchema" and other circumstances occur. Furthermore it's reference to
SolrConfig was removed. (David Smiley)

* SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl

alias is combined with '*' selector
(Michael Lawrence, Munendra S N, Ishan Chattopadhyaya)

* SOLR-13351: Workaround for VELOCITY-908 (Kevin Risden)

* SOLR-13349: High CPU usage in Solr due to Java 8 bug (Erick Erickson)

* SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin (janhoy,

Jason Gerlowski)

* SOLR-13352: Remove risk of deadlock/threadleak when shutting down an

Overseer(TriggerThread). (hossman)

* SOLR-13362: Add 'includeIndexFieldFlags' support to SolrJ LukeRequest (Jason

Gerlowski)

* SOLR-13355: 'all' permission ignored by RuleBasedAuthorizationPlugin in most

cases (Jason Gerlowski, janhoy)

* SOLR-13331: Atomic Update 'remove' operations broken for certain field types in
SolrJ (Thomas Wockinger via Jason Gerlowski)

* SOLR-13388: Fix FileExchangeRateProvider to be a public class, as it appears in

schema.xml (Uwe Schindler)

* SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in

background thread (janhoy, Lorenzo)
* SOLR-13339: Prevent recovery, fetching index being kicked off after SolrCores
already closed (Cao Manh Dat)

* SOLR-13393: Fixed ZkClientClusterStateProvider to prevent risk of leaking

ZkStateReader/threads when
processing concurrent requests during shutdown. This primarily affected tests,
but may have also caused
odd errors/delays when restart/shutting down solr nodes. (hossman)

* SOLR-13336: add maxBooleanClauses (default to 1024) setting to solr.xml,

reverting previous effective
value of Integer.MAX_VALUE-1, to restrict risk of pathalogical query expansion.
(hossman)

* SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node

exists after an exists
check and the Overseer work loop should not allow free spinning the loop when it
hits a KeeperException.
(Mark Miller, Fernandez-Lobbe, Mike Drob)

* SOLR-12371: Editing authorization config via REST API now works in standalone
mode (janhoy)

* SOLR-13408: Cannot start/stop DaemonStream repeatedly, other API improvements

(Erick Erickson)

* SOLR-13281: Fixed NPE in DocExpirationUpdateProcessorFactory (Munendra S N, Tomás

Fernández Löbbe)

* SOLR-13081: In-Place Update doesn't work with route.field (Dr Oleg Savrasov via
Mikhail Khludnev)

* SOLR-13343: Fix spacing issue in browser log (Marcus Eagan via Jason Gerlowski)

* SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields (non-stored,

non-indexed)
(Erick Ericsson, Adrien Grand, Munendra S N, Scott Stults, Ishan Chattopadhyaya)

* SOLR-5970: Return correct status upon collection creation failure (Abraham

Elmahrek, Ishan Chattopadhyaya,
Jason Gerlowski, Kesharee Nandan Vishwakarma)

* SOLR-12291: prematurely reporting not yet finished async Collections API call as
completed
when collection's replicas are colocated at least at one node (Varun Thacker,
Mikhail Khludnev)

* SOLR-13410: Designated overseer wasn't able to rejoin election queue upon restart
(Ishan Chattopadhyaya,
Kesharee Nandan Vishwakarma)

* SOLR-13318: Fix ClassCastException in SolrJ JsonFaceting classes (Munendra S N

via Jason Gerlowski)

* SOLR-13449: SolrClientNodeStateProvider always retries on requesting metrics from

other nodes (Cao Manh Dat)

Improvements
----------------------

* SOLR-12999: Index replication could delete segments before downloading segments

from master if there is not enough
disk space (noble)

* SOLR-13227: Optimizing facet.range.other by avoiding expensive exceptions

(Nikolay Khitrin via Mikhail Khludnev)

* SOLR-9079: Remove commons-lang as a dependency (Kevin Risden)

* SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/)

(Kevin Risden)

* SOLR-13359: Make UpdateHandler support other prefixes (besides hdfs:/) (Kevin

Risden)

* SOLR-13398: Move log line "Processing SSL Credential Provider chain..." from INFO
to DEBUG (janhoy)

* SOLR-13407: Reject update requests sent to non-routed multi collection aliases.

(ab)

* SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches
during SolrCore reload,
added test band-aid for DocValuesNotIndexedTest.

* SOLR-13337: Only request the minimum required number of terms from each shard
when using terms.sort=index and none
are discarded due to terms.min/maxcount (Morten Bøgeskov,Munendra S N via Mikhail
Khludnev)

* SOLR-12167: Throw an exception, instead of just a warning, when unknown atomic

update operation is
encountered (Munendra S N via Ishan Chattopadhyaya)

* SOLR-13394: Switch default GC from CMS to G1 (Ishan Chattopadhyaya, Kesharee

Nandan Vishwakarma, Shawn Heisey,
Uwe Schindler, Erick Ericsson, shalin)

* SOLR-13432: Add .toString methods to BitDocSet and SortedIntDocSet so that

enabling "showItems" on the filter caches
shows some useful information about the values in the cache. (shalin)

* SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-

out lock is not used.
(jefferyyuan, ab)

* SOLR-13348: Speed up collapsing by avoiding scoring of ineligible documents

(Andrzej Wislowski via
Ishan Chattopadhyaya)

Other Changes
----------------------

* SOLR-11763: Upgrade Guava to 25.1-jre (Markus Jelsma, Kevin Risden)

* SOLR-13222: Improve logging in StreamingSolrClients (Peter Cseh via Kevin Risden)

* SOLR-12055: Enable async logging by default. This change improves throughput for
logging. This opens
up a small window where log messages could possibly be lost. If this is
unacceptable, switching back to synchronous
logging can be done by changing the log4j2.xml file, no internal Solr code
changed to make async logging the default.
(Erick Erickson)

* SOLR-12753: Async logging ring buffer and OOM error. When very long messages are
written (1M messages or so),
it can produce an OOM error. Log messages are truncated at 10K via configuration
in the log4j2.xml files.

* SOLR-9763: Remove the workaround implemented for HADOOP-12767 (Kevin Risden)

* SOLR-13060: Improve HdfsAutoAddReplicasIntegrationTest and

HdfsCollectionsAPIDistributedZkTest (Kevin Risden)

* SOLR-13074: MoveReplicaHDFSTest leaks threads, falls into an endless loop,

logging like crazy (Kevin Risden)

* SOLR-9762: Remove the workaround implemented for HADOOP-13346 (Kevin Risden)

* SOLR-7321: Remove reflection in FSHDFSUtils.java (Mike Drob, Kevin Risden)

* SOLR-13268: Clean up any test failures resulting from defaulting to async logging
(Erick Erickson)

* SOLR-13307: Ensure HDFS tests clear System properties they set (Kevin Risden)

* SOLR-13330: Improve HDFS tests (Kevin Risden)

* SOLR-8033: Remove debug if branch in HdfsTransactionLog (Kevin Risden)

* SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud functionality

into a subclass.
(Bar Rotstein, David Smiley)

* SOLR-13342: Remove dom4j from Solr (Kevin Risden)

* SOLR-13335: Upgrade to velocity 2.0 and velocity-tools 3.0 (Kevin Risden)

* SOLR-13112: Upgrade jackson to 2.9.8 (Kevin Risden)

* SOLR-13353: Add SolrCli AuthTool test (Kevin Risden)

* SOLR-13363: Upgrade to ZooKeeper 3.4.14 (Erick Erickson)

* SOLR-12809: Document recommended Java/Solr combinations (Erick Erickson, Jan

Høydahl et.al.)

* SOLR-13366: Clarify 'Invalid stage name' warning logging in AutoScalingConfig

(Christine Poerschke)

* SOLR-13409: Disable HTML directory listings in admin interface to prevent

possible security issues (Uwe Schindler)

* SOLR-12461: Upgrade Dropwizard Metrics to 4.0.5 release. (ab)

* SOLR-13400: Replace Observable pattern in TransientSolrCoreCache (Erick Erickson)

* SOLR-13425: Ref-Guide: Wrong color of text in left column of horizontal

definition list (janhoy)

* SOLR-13423: Upgrade RRD4j to version 3.5. (ab)

* SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined
(Kevin Risden)

================== 8.0.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache ZooKeeper 3.4.13
Jetty 9.4.14.v20181114

Upgrade Notes
----------------------

* Due to the introduction of LIR redesign since Solr 7.3 (SOLR-11702) and the
removing of old LIR implementation in Solr 8.
Rolling updates are not possible unless all nodes must be on Solr 7.3 or higher.
If not updates can be lost.

* Solr nodes can now listen and serve HTTP/2 requests. Most of internal requests
(sent by UpdateShardHandler, HttpShardHandler)
Http2SolrClient is used. Since by default, internal requests are sent by using
HTTP/2,
Solr 8.0 nodes can't talk to old nodes (7.x). However we can follow these steps
to do rolling updates:
- Do rolling updates as normally, but the Solr 8.0 nodes must start with -
Dsolr.http1=true as startup parameter.
By using this parameter internal requests are sent by using HTTP/1.1
- When all nodes are upgraded to 8.0, restart them, this time -Dsolr.http1
parameter should be removed.

* ALPN is not supported in Java 8 or lower version therefore when a node started in
Java 8 or a lower version with SSL enabled,
it will send and can only be able to handle HTTP/1.1 requests. In case of using
SSL Java 9 or latter versions are recommended.

* Custom AuthenticationPlugin must provide its own setup for Http2SolrClient

through
implementing HttpClientBuilderPlugin.setup, if not internal requests can't be
authenticated.

* LUCENE-7996: The 'func' query parser now returns scores that are equal to 0
when a negative value is produced. This change is due to the fact that
Lucene now requires scores to be positive. (Adrien Grand)

* SOLR-11882: SolrMetric registries retained references to SolrCores when closed. A

change of SolrMetricMAnager.registerGauge and
SolrMetricProducer.initializeMetrics
method signatures was required to fix it. Third party components that use this
API
need to be updated. (Eros Taborelli, Erick Erickson, ab)

* LUCENE-8267: Memory codecs have been removed from the codebase (MemoryPostings,
MemoryDocValues). If you used postingsFormat="Memory" or docValuesFormat="Memory"
then either remove it to use the default or experiment with one of the others.
(Dawid Weiss)

* SOLR-12586: The date format patterns used by ParseDateFieldUpdateProcessorFactory

(present in "schemaless mode")
are now interpreted by Java 8's java.time.DateTimeFormatter instead of Joda Time.
The pattern language is very
similar but not the same. Typically, simply update the pattern by changing an
uppercase 'Z' to lowercase 'z' and
that's it. For the current recommended set of patterns in schemaless mode, see
"Schemaless Mode" in the ref guide,
or simply examine the default configSet. Also note that the set of patterns
(formats) here have
expanded from before to subsume those patterns previously handled by the
"extract" contrib (Solr Cell / Tika).
(David Smiley, Bar Rotstein)

* SOLR-12593: The "extraction" contrib (Solr Cell) no longer does any date parsing,
and thus no longer has the
"date.formats" configuration. To ensure date strings are properly parsed, use
ParseDateFieldUpdateProcessorFactory
(an URP) commonly registered with the name "parse-date" in "schemaless mode".
(David Smiley, Bar Rotstein)

* SOLR-12643: Since Http2SolrClient does not support exposing connections related

metrics. These metrics are no longer
available 'QUERY.httpShardHandler.{availableConnections, leasedConnections,
maxConnections, pendingConnections}',
'UPDATE.updateShardHandler.{availableConnections, leasedConnections,
maxConnections, pendingConnections}'

* SOLR-12605: UpdateShardHandler's updateOnlyClient is now a Http2SolrClient

(previous HttpSolrClient). This new
client does not support 'maxConnections','maxConnectionsPerHost' parameters.

* SOLR-12640: HttpShardHandlerFactory's defaultClient is now a Http2SolrClient

(previous HttpSolrClient). This new
client does not support 'maxConnections','maxConnectionsPerHost' parameters.
LBHttpSolrClient.Req and LBHttpSolrClient.Rsp
are marked as deprecated, uses LBSolrClient.Req and LBSolrClient.Rsp instead.

* SOLR-12754: The UnifiedHighlighter hl.weightMatches now defaults to true. If

there are unforseen highlight problems,
this may be the culprit.

* If you explicitly use BM25SimilarityFactory in your schema, the absolute scoring

will be lower due to SOLR-13025.
But ordering of documents will not change in the normal case. Use
LegacyBM25SimilarityFactory if you need to force
the old 6.x/7.x scoring. Note that if you have not specified any similarity in
schema or use the default
SchemaSimilarityFactory, then LegacyBM25Similarity is automatically selected for
'luceneMatchVersion' < 8.0.0.
See also explanation in Reference Guide chapter "Other Schema Elements".

* SOLR-12535: Solr no longer accepts index time boosts in JSON provided to Solr.
This used to be provided like so:
{'id':'1', 'val_s':{'value':'foo', 'boost':2.0}} but will now produce an error.
A object/map structure will now only
be interpreted as a child document or an atomic update; nothing else. A
uniqueKey is currently required on all child
documents to be interpreted as such, though this may change in the future.
(David Smiley)

* SOLR-12633: When JSON data is sent to Solr with nested child documents split
using the "split" parameter, the child
docs will now be associated to their parents by the field/label string used in
the JSON instead of anonymously. Most
users probably won't notice the distinction since the label is lost any way
unless special fields are in the schema.
This choice used to be toggleable with an internal/expert "anonChildDocs"
parameter flag which is now gone.
(David Smiley)

* SOLR-11774: In 'langid' contrib, the LanguageIdentifierUpdateProcessor base class

changed some method signatures.
If you have a custom language identifier implementation you will need to adapt
your code.

* SOLR-9515: Hadoop dependencies have been upgraded to Hadoop 3.2.0 from 2.7.2.
(Mark Miller, Kevin Risden)

* SOLR-5211: Deleting (or updating) documents by their uniqueKey is now scoped to

only consider root documents, not
child/nested documents. Thus a delete-by-id won't work on a child doc (no-op),
and an attempt to update a child doc
by providing a new doc with the same ID would add a new doc (probably erroneous).
Both these actions were and still
are problematic. In-place-updates are safe though. If you want to delete
certain child documents and if you know
they don't themselves have nested children then you must do so with a delete-by-
query technique.

* SOLR-13248: The default replica placement strategy used in Solr has been reverted
to the 'legacy' policy used by Solr
7.4 and previous versions. This is due to multiple bugs in the autoscaling based
replica placement strategy that was
made default in Solr 7.5 which causes multiple replicas of the same shard to be
placed on the same node in addition
to the maxShardsPerNode and createNodeSet parameters being ignored. Although the
default has changed, autoscaling
will continue to be used if a cluster policy or preference is specified or a
collection level policy is in use.
The default replica placement strategy can be changed to use autoscaling again by
setting a cluster property:
curl -X POST -H 'Content-type:application/json' --data-binary '
{
"set-obj-property": {
"defaults" : {
"cluster": {
"useLegacyReplicaAssignment":false
 }
}
}
}' http://$SOLR_HOST:$SOLR_PORT/api/cluster

New Features
----------------------

* SOLR-12591: Expand the set of recognized date format patterns of schemaless mode
to subsume those handled by the
"extract" contrib (Solr Cell / Tika). This is primarily a change in
configuration of the default configSet for more
patterns, but also included enabling "lenient" parsing in
ParseDateFieldUpdateProcessorFactory. The default
locale was changed from ROOT to en_US since well-known patterns assume this
locale.
(David Smiley, Bar Rotstein)

* SOLR-12879: MinHash query parser that builds queries providing a measure of

Jaccard similarity (Andy Hind via Tommaso Teofili)

* SOLR-12593: The default configSet now includes an "ignored_*" dynamic field.

(David Smiley)

* SOLR-12799: Allow Authentication Plugins to intercept internode requests on a

per-request basis.
The BasicAuth plugin now supports a new parameter 'forwardCredentials', and when
set to 'true',
user's BasicAuth credentials will be used instead of PKI for client initiated
internode requests. (janhoy, noble)

* SOLR-12791: Add Metrics reporting for AuthenticationPlugin (janhoy)

* SOLR-12730: Implement staggered SPLITSHARD requests in IndexSizeTrigger. (ab)

* SOLR-12639: Umbrella JIRA for adding support HTTP/2 (Cao Manh Dat)

* SOLR-12768, SOLR-13129: Improved nested document support, and enabled in the

default schema with the presence of _nest_path_.
When this field is present, certain things happen automatically. An internal URP
is automatically used to populate
it. The [child] (doc transformer) will return a hierarchy with relationships; no
params needed. The relationship
path is indexed for use in queries (can be disabled if not needed). Also, child
documents needn't provide a uniqueKey
value as Solr will supply one automatically by concatenating a path to that of
the parent document's key.
(David Smiley, Moshe Bla).

* SOLR-13134: Allow the knnRegress Stream Evaluator to more easily perform

bivariate regression.
(Joel Bernstein)

* SOLR-13104: Add natural and repeat Stream Evaluators (Joel Bernstein)

* SOLR-13147: Add movingMAD Stream Evaluator (Joel Bernstein)

* SOLR-13155: Add command-line option for testing autoscaling configurations. (ab)

* SOLR-13241: Add 'autoscaling' tool support to solr.cmd (Jason Gerlowski)

* SOLR-11126: New Node-level health check handler at /admin/info/healthcheck and

/node/health paths that
checks if the node is live, connected to zookeeper and not shutdown. (Anshum
Gupta, Amrit Sarkar, shalin)

Bug Fixes
----------------------

* SOLR-13058: Fix block that was synchronizing on the wrong collection in

OverseerTaskProcessor (Gus Heck)

* SOLR-11774: langid.map.individual now works together with langid.map.keepOrig.

Also the detectLanguage() API
is changed to accept a Reader allowing for more memory efficient implementations
(janhoy)

* SOLR-13126: Query boosts were not being combined correctly for documents where
not all boost queries
matched (Alan Woodward, Mikhail Khludnev)

* SOLR-13248: Autoscaling based replica placement is broken out of the box. Solr
7.5 enabled autoscaling based replica
placement by default but in the absence of default cluster policies, autoscaling
can place more than 1 replica of the
same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not
respected. Due to these reasons,
this issue reverts the default replica placement policy to the 'legacy'
assignment policy that was the default until
Solr 7.4. (Gus Heck, Andrzej Bialecki, Bram Van Dam, shalin)

* SOLR-13255 : ClasscastException when URPs try to read a String field which

returns a ByteArrayUTF8CHarSequence . This is a regression
in release 7.7 (noble)

*SOLR-13299: Fix Windows startup script to disable HTTP/2 if TLS is enabled on Java
8. (Uwe Schindler)

Improvements
----------------------

* SOLR-5211: If _root_ is defined in the schema, it is now always populated

automatically. This allows documents with
children to be updated with a document that does not have children, whereas before
it would break block-join queries.
If you don't use nested documents then _root_ can be removed from the schema. (Dr
Oleg Savrasov, Moshe Bla,
David Smiley, Mikhail Khludnev)

* SOLR-6117: The response format has changed slightly for ReplicationHandler error-
cases. All errors now have a non-200
'status' field, a 'message' field giving more details on the error, and an
optional 'exception' field. (Shalin Mangar,
Jason Gerlowski)

* SOLR-12888: The NestedUpdateProcessor (which populates internal fields for nested

child docs) is now auto-registered
to run immediately prior to RunUpdateProcessor. If the schema has no _nest_*_
fields then it's a no-op.
RunUpdateProcessorFactory looks up a special/internal URP chain "_preRun_" which
is implicitly defined but could be
defined by an app for customization if desired. (David Smiley)

Optimizations
----------------------

* SOLR-12725: ParseDateFieldUpdateProcessorFactory should reuse ParsePosition. (ab)

* SOLR-13025: Due to LUCENE-8563, the BM25Similarity formula no longer includes the

(k1+1) factor in the numerator
This gives a lower absolute score but doesn't affect ordering, as this is a
constant factor which is the same
for every document. Use LegacyBM25SimilarityFactory if you need the old 6.x/7.x
scoring. See also upgrade notes (janhoy)

* SOLR-13130: during the ResponseBuilder.STAGE_GET_FIELDS directly copy string

bytes and avoid creating String Objects (noble)

Other Changes
----------------------

* SOLR-12614: Make "Nodes" view the default in AdminUI "Cloud" tab (janhoy)

* SOLR-12586: Upgrade ParseDateFieldUpdateProcessorFactory (present in "schemaless

mode") to use Java 8's
java.time.DateTimeFormatter instead of Joda time (see upgrade notes). "Lenient"
is enabled. Removed Joda Time dependency.
(David Smiley, Bar Rotstein)

* SOLR-5163: edismax now throws an exception when qf refers to a nonexistent field

(Charles Sanders, David Smiley)

* SOLR-12805: Store previous term (generation) of replica when start recovery

process (Cao Manh Dat)

* SOLR-12652: Remove SolrMetricManager.overridableRegistryName method (Peter

Somogyi via David Smiley)

* LUCENE-8513: SlowCompositeReaderWrapper now uses MultiTerms directly instead of

MultiFields (David Smiley)

* SOLR-11812: Remove backward compatibility of old LIR implementation in 8.0 (Cao

Manh Dat)

* SOLR-12620: Remove the Admin UI Cloud -> Graph (Radial) view (janhoy)

* SOLR-12775: LowerCaseTokenizer is deprecated, and should be replaced by

LetterTokenizer and
LowerCaseFilter (Alan Woodward)

* SOLR-13036: Fix retry logic in JettySolrRunner (Gus Heck)

* SOLR-12535: Solr no longer accepts index time boosts in JSON provided to Solr.
(David Smiley)
* SOLR-13086: Improve the error message reported by DocumentObjectBinder when a
setter is not found (Gus Heck)

* SOLR-12365: Renamed class Config to XmlConfigFile (David Smiley)

* SOLR-9515: Hadoop dependencies have been upgraded to Hadoop 3.2.0 from 2.7.2.
(Mark Miller, Kevin Risden)

================== 7.7.3 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache ZooKeeper 3.4.14
Jetty 9.4.14.v20181114

Bug fixes
----------------------

* SOLR-13779: Use the safe fork of simple-xml for clustering contrib. (Dawid Weiss,
Sachin Pattan)

* SOLR-13718: SPLITSHARD (async) with failures in underlying sub-operations can

result in data loss (Ishan Chattopadhyaya)

* SOLR-12291: prematurely reporting not yet finished async Collections API call as
completed
when collection's replicas are colocated at least at one node (Varun Thacker,
Mikhail Khludnev)

* SOLR-13828: Improve ExecutePlanAction error handling.

* SOLR-13472: Forwarded requests should skip authorization on receiving nodes

(Ishan Chattopadhyaya)

* SOLR-13793: HttpSolrCall now maintains internal request count (_forwardedCount)

for remote queries and limits them to
the number of replicas. This avoids making too many cascading calls to remote
servers, which, if not restricted, can
bring down nodes containing the said collection (Kesharee Nandan Vishwakarma,
Ishan Chattopadhyaya)

* SOLR-13971: Velocity response writer's resource loading now possible only through
startup parameters. Also, removed velocity
response writer from _default configset. (Ishan Chattopadhyaya, Sachin Pattan)

* SOLR-14025: VelocityResponseWriter has been hardened - only trusted configsets

can render configset provided
templates and rendering templates from request parameters has been removed.

* SOLR-13158: DataImportHandler: Added enable.dih.dataConfigParam system property

to toggle whether the dataConfig param
 is permitted. (David Smiley, janhoy, Tomás Fernández Löbbe)

* SOLR-14259: Back port javabin performance regression fixes from SOLR-14013

(noble)

================== 7.7.2 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache ZooKeeper 3.4.14
Jetty 9.4.14.v20181114

Upgrade Notes
----------------------

* SOLR-13234: The Prometheus Metric Exporter now collects metrics from Solr at a
fixed schedule that is controlled
by the exporter itself. The default interval is 60s but it can be changed by
specifying the --scrape-interval
command line parameter to the tool. The collected metrics are cached and returned
for any requests coming in before
the next scheduled collection. This can impact the freshness of the metrics if
prometheus is configured to query the
tool more frequently.

* SOLR-12708: When requesting the status of an async request via REQUESTSTATUS

collections API, the response will
include the list of internal async requests (if any) in the "success" or "failed"
keys (in addition
to them being included outside those keys for backwards compatibility). See SOLR-
12708 for more
details

Bug fixes
----------------------

* SOLR-13234: Prometheus Metric Exporter not threadsafe. This changes the

prometheus exporter to collect metrics
from Solr on a fixed interval controlled by this tool and prevents concurrent
collections. This change also improves
performance slightly by using the cluster state instead of sending multiple HTTP
requests to each node to lookup
all the cores.
(Danyal Prout via shalin)

* SOLR-12708: Async collection actions should not hide internal failures (Mano
Kovacs, Varun Thacker, Tomás Fernández Löbbe)

* SOLR-11876: In-place update fails when resolving from Tlog if schema has a
required field (Justin Deoliveira, janhoy,
Ishan Chattopadhyaya)
* SOLR-13349: High CPU usage in Solr due to Java 8 bug (Erick Erickson)

* SOLR-13355: 'all' permission ignored by RuleBasedAuthorizationPlugin in most

cases (Jason Gerlowski, janhoy)

* SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin (janhoy,

Jason Gerlowski)

* SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in

background thread (janhoy, Lorenzo)

* SOLR-13331: Atomic Update 'remove' operations broken for certain field types in
SolrJ (Thomas Wockinger via Jason Gerlowski)

* SOLR-13285: Updates with enum fields and javabin cause ClassCastException (noble)

* SOLR-12371: Editing authorization config via REST API now works in standalone
mode (janhoy)

* SOLR-13281: Fixed NPE in DocExpirationUpdateProcessorFactory (Munendra S N, Tomás

Fernández Löbbe)

* SOLR-13409: Disable HTML directory listings in admin interface to prevent

possible security issues (Uwe Schindler)

* SOLR-13126: Query boosts were not being combined correctly for documents where
not all boost queries
matched (Alan Woodward, Mikhail Khludnev)

* SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined
(Kevin Risden)

* SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-

out lock is not used. (ab)

* SOLR-13449: SolrClientNodeStateProvider always retries on requesting metrics from

other nodes (Cao Manh Dat)

* SOLR-13112: Upgrade jackson to 2.9.8 (Kevin Risden)

* SOLR-13410: Designated overseer wasn't able to rejoin election queue at head upon
restart (Ishan Chattopadhyaya,
Kesharee Nandan Vishwakarma)

* SOLR-13254: Correct message that is logged in solrj's ConnectionManager when an

exception
occurred while reconnecting to ZooKeeper. (hu xiaodong via Christine Poerschke)

* SOLR-13252: Fix an NPE when setting a "policy" property for an existing

collection (ab)

* SOLR-13398: Move log line "Processing SSL Credential Provider chain..." from INFO
to DEBUG (janhoy)

* SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node

exists after an exists
check and the Overseer work loop should not allow free spinning the loop when it
hits a KeeperException.
(Mark Miller, Fernandez-Lobbe, Mike Drob)
* SOLR-13366: Clarify 'Invalid stage name' warning logging in AutoScalingConfig
(Christine Poerschke)

* SOLR-13352: Remove risk of deadlock/threadleak when shutting down an

Overseer(TriggerThread). (hossman)

* SOLR-13229: Cleanup replicasMetTragicEvent after all types of exception (Tomás

Fernández Löbbe)

Other Changes
----------------------

* SOLR-13363: Upgrade to ZooKeeper 3.4.14 (Erick Erickson)

================== 7.7.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache ZooKeeper 3.4.13
Jetty 9.4.14.v20181114

Upgrade Notes
----------------------

* SOLR-13248: The default replica placement strategy used in Solr has been
reverted to the 'legacy' policy used by Solr
7.4 and previous versions. This is due to multiple bugs in the autoscaling based
replica placement strategy that was
made default in Solr 7.5 which causes multiple replicas of the same shard to be
placed on the same node in addition
to the maxShardsPerNode and createNodeSet parameters being ignored. Although the
default has changed, autoscaling
will continue to be used if a cluster policy or preference is specified or a
collection level policy is in use.
The default replica placement strategy can be changed to use autoscaling again
by setting a cluster property:
curl -X POST -H 'Content-type:application/json' --data-binary '
{
"set-obj-property": {
"defaults" : {
"cluster": {
"useLegacyReplicaAssignment":false
}
}
}
}' http://$SOLR_HOST:$SOLR_PORT/api/cluster

Bug Fixes
----------------------

* SOLR-13255 : ClasscastException when URPs try to read a String field which

returns a ByteArrayUTF8CHarSequence . This is a regression
 in release 7.7 (noble)

* SOLR-13248: Autoscaling based replica placement is broken out of the box. Solr
7.5 enabled autoscaling based replica
placement by default but in the absence of default cluster policies, autoscaling
can place more than 1 replica of the
same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not
respected. Due to these reasons,
this issue reverts the default replica placement policy to the 'legacy'
assignment policy that was the default until
Solr 7.4. (Gus Heck, Andrzej Bialecki, Bram Van Dam, shalin)

================== 7.7.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache ZooKeeper 3.4.13
Jetty 9.4.14.v20181114

Upgrade Notes
----------------------

* SOLR-12770: The 'shards' parameter handling logic changes to use a new config
element to determine what hosts can be
requested. Please see Apache Solr Reference Guide chapter "Distributed Requests"
for details, as well as SOLR-12770.

New Features
----------------------

* SOLR-12839: JSON 'terms' Faceting now supports a 'prelim_sort' option to use when
initially selecting
the top ranking buckets, prior to the final 'sort' option used after refinement.
(hossman)

* SOLR-7896: Add a login page to Admin UI, with initial support for Basic Auth
(janhoy)

* SOLR-13116: Add Admin UI login support for Kerberos (janhoy, Jason Gerlowski)

* SOLR-12770: Make it possible to configure a host whitelist for distributed search

(Christine Poerschke, janhoy, Erick Erickson, Tomás Fernández Löbbe)

* SOLR-12373: Add a "tombstoneConfig" option to DocBasedVersionConstraintsProcessor

that allows
users to configure which fields/values to add to tombstone documents. This can be
useful to
make sure tombstone documents include fields that are marked as required in the
schema
(Tomás Fernández Löbbe)

* SOLR-12984: The search Streaming Expression should properly support and push down
paging
when using the /select handler (Joel Bernstein)

* SOLR-13088: Add zplot Stream Evaluator to plot math expressions in Apache

Zeppelin (Joel Bernstein)

Bug Fixes
----------------------

* SOLR-12546: CVSResponseWriter omits useDocValuesAsStored=true field when fl=*

(Munendra S N via Mikhail Khludnev)

* SOLR-12933: Fix SolrCloud distributed commit. (Mark Miller)

* SOLR-13014: URI Too Long with large streaming expressions in SolrJ (janhoy)

* SOLR-13066: A failure while reloading a SolrCore can result in the SolrCore not
being closed. (Mark Miller)

* SOLR-11296: Spellcheck parameters not working in new UI (Matt Pearce via janhoy)

* SOLR-10975: New Admin UI Query does not URL-encode the query produced in the URL
box (janhoy)

* SOLR-13072: Management of markers for nodeLost / nodeAdded events is broken. This

bug could have caused
some events to be lost if they coincided with an Overseer leader crash. (ab)

* SOLR-13080: The "terms" QParser's "automaton" method semi-required that the input
terms/IDs be sorted. This
query parser now does this. Unclear if this is a perf issue or actual bug.
(Daniel Lowe, David Smiley)

* SOLR-13082: A trigger that creates trigger events more frequently than the cool
down period can starve other triggers.
This is mitigated to some extent by randomly choosing the trigger to resume after
cool down. It is recommended that
scheduled triggers not be used for very frequent operations to avoid this
problem.
(ab, shalin)

* SOLR-12514: Rule-base Authorization plugin skips authorization if querying node

does not have collection replica (noble)

* SOLR-11853: Solr installer fails on SuSE linux (Markus Mandalka via janhoy)

* SOLR-12237: Fix incorrect SOLR_SSL_KEYSTORE_TYPE variable in solr start script

(janhoy, Joel Bernstein)

* SOLR-13053: NodeAddedTrigger and NodeLostTrigger do not reserve added/removed

time populated by restoreState
(Cao Manh Dat)

* SOLR-13137: NPE when /admin/zookeeper/status endpoint hit in standalone mode

(janhoy)

* SOLR-13091: REBALANCELEADERS is broken (Erick Erickson)

* SOLR-11998: RebalanceLeaders API broken response format with wt=JSON (Erick

Erickson)

* SOLR-9735: Fix v2 API for AutoscalingHistoryHandler. (ab)

* SOLR-13168: Fixed a bug in TestInjection that caused test only code to be invoked
when TLOG replicas
recieved commits if java assertions were enabled. (hossman)

Improvements
----------------------

* SOLR-12881: Remove unneeded import statements (Peter Somogyi via Erick Erickson)

* SOLR-12992: When using binary format, ExportWriter to directly copy BytesRef

instead of
creating new String (noble)

* SOLR-12898: Replace cluster state polling with ZkStateReader#waitFor. (Mark

Miller)

* SOLR-12897: Introduce AlreadyClosedException to clean up silly close / shutdown

logging. (Mark Miller)

* SOLR-12896: Introduce more checks for shutdown and closed to improve clean close
and shutdown. (Mark Miller)

* SOLR-12804: Remove static modifier from Overseer queue access. (Mark Miller)

* SOLR-12833: Add configurable timeout to VersionBucket lock. (Jeffery Yuan, Mark

Miller)

* SOLR-12885: BinaryResponseWriter (javabin format) should directly copy from

BytesRef to output (noble)

* SOLR-12973: Admin UI "Nodes" view support for replica* replica names. (Daniel
Collins, Christine Poerschke, janhoy)

* SOLR-13090: All shipped configurations still have `maxBooleanClauses` default to

1024. But if the
`solr.max.booleanClauses` sysprop is specified, that will override the 1024
default. This enables users to
update this property across the board more easily. (Jason Gerlowski)

* SOLR-12983: JavabinLoader should avoid creating String Objects and create

UTF8CharSequence fields from byte[] (noble)

* SOLR-13016: Computing suggestions when policy have "#EQUAL" or "#ALL" rules take
too long (noble)

* SOLR-13029: solr.hdfs.buffer.size can be configured for HdfsBackupRepository for

better performance (Tim Owen via Mikhail Khludnev)

* SOLR-13156: support facet.sort for facet.field={!terms=foo,bar}field. (Konstantin

Perikov via Mikhail Khludnev)

* SOLR-13146: Allow derivatives to be computed for the oscillate Stream Evaluator

(Joel Bernstein)

Other Changes
----------------------

* SOLR-12972: deprecate unused SolrIndexConfig.luceneVersion (Christine Poerschke)

* SOLR-12801: Make massive improvements to the tests. (Mark Miller)

* SOLR-12923: The new AutoScaling tests are way too flaky and need special
attention. (Mark Miller)

* SOLR-12932: ant test (without badapples=false) should pass easily for developers.
(Mark Miller)

* SOLR-13036: Fix retry logic in JettySolrRunner (Gus Heck)

* SOLR-12727: Upgrade ZooKeeper dependency to 3.4.13 (Kevin Risden, Erick Erickson,

Cao Manh Dat)

* SOLR-13086: Improve the error message reported by DocumentObjectBinder when a

setter is not found (Gus Heck)

================== 7.6.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.19.1
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache ZooKeeper 3.4.11
Jetty 9.4.11.v20180605

Upgrade Notes
----------------------

* SOLR-12767: The min_rf parameter is no longer needed, Solr will always return the
achieved replication factor (rf)
in the response header.

* SOLR-12827: The cluster wide defaults structure has changed from

{collectionDefaults: {nrtReplicas : 2}} to
{defaults : {collection : {nrtReplicas : 2}}}. The old format continues to be
supported and can be read from
ZK as well as written using the V2 set-obj-property syntax but it is deprecated
and will be removed in Solr 9.
We recommend that users change their API calls to use the new format going
forward.

* SOLR-12739: Autoscaling policy framework is now used as the default strategy to

select the nodes on which
new replicas or replicas of new collections are created. Previously, the
maxShardsPerNode parameter was not allowed
on collections when autoscaling policy was configured. Also if an autoscaling
policy was configured then the default
was to set an unlimited maxShardsPerNode automatically. Now the maxShardsPerNode
parameter is always
allowed during collection creation and maxShardsPerNode should be set correctly
(if required) regardless of whether
autoscaling policies are in effect or not. The default value of maxShardsPerNode
continues to be 1 as before. It can
be set to -1 during collection creation to fall back to the old behavior of
unlimited maxShardsPerNode when using
autoscaling policy.

* SOLR-12861: Added a Solr factory for ByteBuffersDirectory, which will replace

deprecated RAMDirectory in Solr 9.0.

New Features
----------------------

* SOLR-6280: {!collapse}: if you attempt to use CollapseQParser on a field that is

multi-valued, you will now get an
error. Previously, the collapsing behavior was unreliable and undefined despite
no explicit error.
(Munendra S N, David Smiley)

* SOLR-9317: ADDREPLICA command should be able to add more than one replica to a
collection,shard at a time.
The API now supports 'nrtReplicas', 'tlogReplicas', 'pullReplicas' parameters as
well 'createNodeSet' parameter.
As part of this change, the CREATESHARD API now delegates placing replicas
entirely to the ADDREPLICA command
and uses the new parameters to add all the replicas in one API call. (shalin)

* SOLR-11522: /autoscaling/suggestions now include rebalance options as well even

if there are no violations (noble)

* SOLR-12822: /autoscaling/suggestions to include suggestion to add-replica for

lost replicas (noble)

* SOLR-12815: Implement maxOps limit for IndexSizeTrigger. (ab)

* SOLR-12843: Implement a MultiContentWriter in SolrJ to post multiple

files/payload at once (noble)

* SOLR-12780: Add support for Leaky ReLU and TanH activations in contrib/ltr
NeuralNetworkModel class.
(Kamuela Lau, Christine Poerschke)

* SOLR-12846: Added support for "host" variable in autoscaling policy rules (noble)

* SOLR-5004: Splitshard collections API now supports splitting into more than 2
sub-shards directly i.e. by providing a
numSubShards parameter (Christine Poerschke, Anshum Gupta)

* SOLR-12754: The UnifiedHighlighter has a new hl.weightMatches param defaulting to

false (will be true in 8.0). It's
the highest query accuracy mode, and furthermore phrase queries are highlighted
as one. (David Smiley)

* SOLR-12828: Add oscillate Stream Evaluator to support sine wave analysis. (Joel
Bernstein)

* SOLR-11907: Add convexHull and associated geometric Stream Evaluators. (Joel

Bernstein)
* SOLR-12811: Add enclosingDisk and associated geometric Stream Evaluators. (Joel
Bernstein)

* SOLR-12840: Add pairSort Stream Evaluator (Joel Bernstein)

* SOLR-12862: Add log10 Stream Evaluator and allow the pow Stream Evaluator to
accept a vector of exponents (Joel Bernstein)

* SOLR-12942: Add an option in IndexSizeTrigger to select the split shard method.

(ab)

* SOLR-12938: Cluster Status returns results for aliases, instead of throwing

exceptions (Gus Heck)

* SOLR-11997: Suggestions API/UI should show an entry where a violation could not
be resolved (noble)

* SOLR-12971: Add pivot Stream Evaluator to pivot facet co-occurrence counts into a
matrix (Joel Bernstein)

* SOLR-12795: Introduce 'rows' and 'offset' parameter in FacetStream (Joel

Bernstein, Amrit Sarkar, Varun Thacker)

* SOLR-11572: Add recip Stream Evaluator to support reciprocal transformations

(Joel Bernstein)

* SOLR-12936: Allow percentiles Stream Evaluator to accept an array of percentiles

to calculate (Joel bernstein)

* SOLR-12829: Add plist (parallel list) Streaming Expression (Joel Bernstein)

* SOLR-12975: Add ltrim and rtrim Stream Evaluators (Joel Bernstein)

* SOLR-12962: Added a new 'uninvertible' option for fields and fieldtypes. This
defaults to 'true' for
backcompat allowing a FieldCache to be built for indexed fields as needed, but
users are encouraged
to set this to false (using docValues as needed) to reduce the risk of large
fluxuations in heap
size due to unexpected attempts to sort/facet/function on non-docValue fields.
(hossman)

Other Changes
----------------------

* SOLR-12762: Fix javadoc for SolrCloudTestCase.clusterShape() method and add a

method that validates only against
Active slices (Anshum Gupta)

* SOLR-12756: Refactor Assign and extract replica placement strategies out of it.
Now, assignment is done with the help
of a builder class instead of calling a method with large number of arguments.
The number of special cases that had
to be handled have been cut down as well. (shalin)

* SOLR-12827: Migrate cluster wide defaults syntax in cluster properties to a

nested structure. The structure has
changed from {collectionDefaults: {nrtReplicas : 2}} to {defaults : {collection :
{nrtReplicas : 2}}}.
(ab, shalin)

* SOLR-12835: Document statistics exposed by the Query Result Cache when maxRamMB
is configured. (shalin)

* SOLR-12423: Upgrade to Tika 1.19.1 when available (Tim Allison via Erick
Erickson)

* SOLR-12793: Move TestCloudJSONFacetJoinDomain and TestCloudJSONFacetSKG to the

facet test package (Varun Thacker)

* SOLR-12861: Add Solr factory for ByteBuffersDirectory.

* SOLR-12746: Simplify the Ref Guide HTML structure and use semantic HTML tags
where possible. Adds new template files
for Asciidoctor HTML conversion. Building the HTML version now requires the Slim
gem. (Cassandra Targett)

* SOLR-12956: Add Javadoc @since tag to Analyzer component classes (Alexandre

Rafalovitch)

* SOLR-12966: Add Javadoc @since tag to URP classes (Alexandre Rafalovitch)

* SOLR-12600: Fix parameter names in Solr JSON documentation (Alexandre

Rafalovitch)

* SOLR-12497: Add documentation to use Hadoop credential provider-based

keystore/trustsore.
(Mano Kovacs, Cassandra Targett)

* SOLR-13006: ZkNodeProps to be able to load from both javabin and JSON (noble)

Bug Fixes
----------------------

* SOLR-12803: Ensure ConcurrentUpdateSolrClient sends documents to correct

collection (Jason Gerlowski)

* SOLR-11836: FacetStream works with bucketSizeLimit of -1 which will fetch all the
buckets.
(Alfonso Muñoz-Pomer Fuentes, Amrit Sarkar via Varun Thacker)

* SOLR-12776: Setting of TMP in solr.cmd causes invisibility of Solr to JDK tools

(Petr Bodnar via Erick Erickson)

* SOLR-12648: Autoscaling framework based replica placement is not used unless a

policy is specified or
non-empty cluster policy exists. (shalin)

* SOLR-12750: Migrate API should lock the collection instead of shard. (shalin)

* SOLR-12767: When using min_rf, shard leader skipped bad replicas instead of
marking them for recovery, this could
create inconsistencies between replicas of the same shard. min_rf parameter is
now deprecated, and even if provided
replicas that don't ack an update from the leader will be marked for recovery.
(Tomás Fernández Löbbe)
* SOLR-12814: Metrics history causing "HttpParser URI is too large >8192" when many
collections (janhoy)

* SOLR-12836: ZkController creates a cloud solr client with no connection or read

timeouts. Now the http client
created by the update shard handler is used instead. (shalin)

* SOLR-12729: SplitShardCmd should lock the parent shard to prevent parallel

splitting requests. (ab)

* SOLR-12851: Improvements and fixes to let and select Streaming Expressions (Joel
Bernstein)

* SOLR-12874: Java 9+ GC Logging filesize parameter should use a unit. (Tim

Underwood via Uwe Schindler)

* SOLR-12868: Request forwarding for v2 API is broken (noble)

* SOLR-7557: Fix parsing of child documents using queryAndStreamResponse (Marvin

Bredal Lillehaug/Stian Østerhaug via janhoy)

* SOLR-12875: fix ArrayIndexOutOfBoundsException when unique(field) or

uniqueBlock(_root_) is
used with DVHASH method in json.facet. (Tim Underwood via Mikhail Khludnev)

* SOLR-12954: fix facet.pivot refinement bugs when using facet.sort=index and

facet.mincount>1 (hossman)

* SOLR-12023: Autoscaling policy engine shuffles replicas needlessly (noble)

* SOLR-12243: Edismax missing phrase queries when phrases contain multiterm

synonyms
(Elizabeth Haubert, Alessandro Benedetti, Uwe Schindler, Steve Rowe)

* SOLR-12977: Autoscaling tries to fetch metrics from dead nodes (noble)

* SOLR-12978: In autoscaling NPE thrown for nodes where value is absent (noble)

Improvements
----------------------

* SOLR-12767: Solr now always includes in the response of update requests the
achieved replication factor
(Tomás Fernández Löbbe)

* SOLR-12782: UninvertingReader wrapping is now a bit more lightweight: Does not

create FieldInfo for fields that
can't be uninverted (saves mem) and can avoid wrapping the reader altogether if
there's nothing to uninvert.
IndexSchema.getUninversionMap refactored to getUninversionMapper and no longer
merges FieldInfos. (David Smiley)

* SOLR-12739: Make autoscaling policy based replica placement the default strategy
for placing replicas. (shalin)

* SOLR-12806: use autoscaling policies with strict=false to prioritize node

allocation (noble)

* SOLR-10981: Support for stream.url or stream.file pointing to gzipped data. It's

detected by either a content
encoding header or file extension. (Andrew Lundgren via David Smiley, Jan
Høydahl)

* SOLR-12892: MapWriter to use CharSequence instead of String (noble)

* SOLR-12882: Eliminate excessive lambda allocation in json facets

FacetFieldProcessorByHashDV (Tim Underwood)

* SOLR-12699: Make contrib/ltr LTRScoringModel immutable and cache its hashCode.

(Stanislav Livotov, Edward Ribeiro, Christine Poerschke)

* LUCENE-8557: Some internal LeafReader.getFieldInfos implementations were being

re-computed on-demand instead of
once up front leading to some slowdowns in places like JSON Facets and field
collapsing. (Tim Underwood, David Smiley)

* SOLR-12964: Json Facets: use DocValuesIterator advanceExact() instead of

advance() in FacetFieldProcessorByHashDV and
UniqueSinglevaluedSlotAcc. (Tim Underwood)

* SOLR-12880: Json Facets: Show the FacetProcessor class name instead of the
FacetRequest in the JSON Facets debug-trace
output (Tim Underwood)

================== 7.5.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.18
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache ZooKeeper 3.4.11
Jetty 9.4.11.v20180605

Upgrade Notes
----------------------

* The data-driven schema no longer returns the *_str field-copy of text fields by
default. The *_str field is still
created and can be used for e.g. sorting, but to retrieve it you now need to
explicitly list it in "fl" (SOLR-12350)

* LUCENE-7976: TieredMergePolicy now respects maxSegmentSizeMB by default when

executing
findForcedMerges (optimize) and findForcedDeletesMerges (expungeDeletes) (Erick
Erickson)

* SOLR-12395: SignificantTermsQParserPlugin's name is now 'significantTerms' and

its old name 'sigificantTerms' is deprecated.

* SOLR-11694: Extremely outdated UIMA contrib module has been removed (Alexandre
Rafalovitch)

* SOLR-12008: The configuration file for log4j2.xml is now in

../server/resources/log4j2.xml. All other copies have been removed.
 WINDOWS USERS: This JIRA corrects a bug in the start scripts that put example
logs under ...\server, solr.log is
now under ...\example. (Erick Erickson)

* SOLR-12398: The heatmap facet response structure, when returned in JSON, changed
from an even/odd name/value array to
an object style. Although the object style makes more sense, this was an
overlooked back-compat break; sorry.

New Features
----------------------

* SOLR-11865: The QueryElevationComponent now has a useConfiguredElevatedOrder

setting. When multiple docs are elevated,
this specifies whether their relative order should be the order in the
configuration file or if not then should they
be subject to whatever the sort criteria is. Additionally, QEC was extensively
refactored to be more extensible.
(Bruno Roustant, David Smiley)

* SOLR-12474: Add an UpdateRequest Object that implements

RequestWriter.ContentWriter (noble)

* SOLR-12361: Allow nested child documents to be in field values of a

SolrInputDocument as an alternative to
add/get ChildDocuments off to the side. The latter is now referred to as
"anonymous" child documents as opposed to
"labelled" (by the field name). Anonymous child docs might be deprecated in the
future. This is an internal change
that should work for javabin/SolrJ; separate issues will address XML & JSON
formats populating nested docs in this way.
AddUpdateCommand and it's relationship with DirectUpdateHandler2 was reworked
substantially. (Moshe Bla, David Smiley)

* SOLR-12362: Uploading docs in JSON now supports child documents as field values,
thus providing a label to the
relationship instead of the current "anonymous" relationship. Use of this
experimental feature sometimes requires a
anonChildDocs=false parameter until Solr 8 due to syntax ambiguities. (Moshe
Bla, David Smiley)

* SOLR-12485: Uploading docs in XML now supports child documents as field values,
thus providing a label to the
relationship instead of the current "anonymous" relationship. (Moshe Bla, David
Smiley)

* SOLR-12441: (EXPERIMENTAL) New NestedUpdateProcessorFactory (URP) to populate

special fields _nest_parent_ and
_nest_path_ of nested (child) documents. It will generate a uniqueKey of nested
docs if they were blank too.
(Moshe Bla, David Smiley)

* SOLR-12519: The [child] transformer now returns a nested child doc structure
(attached as fields if provided this way)
provided the schema has the _nest_path_ field. This is part of a broader
enhancement of nested docs.
(Moshe Bla, David Smiley)
* SOLR-12722: The [child] transformer now takes an 'fl' param to specify which
fields to return. It will evaluate
doc transformers if present. In 7.5 a missing 'fl' defaults to the current
behavior of all fields, but in 8.0
defaults to the top/request "fl". (Moshe Bla, David Smiley)

* SOLR-11578: Solr 7 Admin UI (Cloud > Graph) should reflect the Replica type to
give a more accurate representation
of the cluster. (Rhoit Singh via Erick Erickson)

* SOLR-12507: Modify collection API should support un-setting properties. (shalin)

* SOLR-12506: Add SolrJ support for the modify collection API. (shalin)

* SOLR-12398: The JSON Facet API now supports type=heatmap facets, just as classic
faceting does. (David Smiley)

* SOLR-11985: Support percentage values in replica attribute in autoscaling policy

(noble)

* SOLR-12511: Support non integer values for replica in autoscaling policy (noble)

* SOLR-12517: Support range values for replica in autoscaling policy (noble)

* SOLR-12530: Ability to disable configset upload via -

Dconfigset.upload.enabled=false startup parameter
(Ishan Chattopadhyaya)

* SOLR-12495: An #EQUAL function for replica in autoscaling policy to equally

distribute replicas (noble)

* SOLR-11986: Allow percentage in freedisk attribute in autoscaling policy rules

(noble)

* SOLR-12522: Support a runtime function `#ALL` for 'replica' in autoscaling

policies (noble)

* SOLR-12567: JSON Facet "functions" now support an extended "type:func" syntax,

similar to other types
of facets. This also allows additional local params to be specified for if the
aggregation function
can take advantage of them. (hossman)

* SOLR-12581: the JSON Facet 'relatedness()' aggregate function now supports a

'min_popularity' option
using the extended type:func syntax (hossman)

* SOLR-12536: autoscaling policy support to equally distribute replicas on the

basis of arbitrary properties (noble)

* SOLR-8207: Add "Nodes" view to the Admin UI "Cloud" tab, listing nodes and key
metrics (janhoy)

* SOLR-7767: "ZK Status" sub menu under "Cloud" tab to see status of zookeeper
ensemble (janhoy)

* SOLR-11990: Make it possible to co-locate replicas of multiple collections

together in a node. A collection may be
 co-located with another collection during collection creation time by specifying
a 'withCollection' parameter. It can
also be co-located afterwards by using the modify collection API. The co-location
guarantee is enforced regardless of
future cluster operations whether they are invoked manually via the Collection
API or by the Autoscaling framework.
(noble, shalin)

* SOLR-12402: Factor out SolrDefaultStreamFactory class. (Christine Poerschke)

* SOLR-12592: support #EQUAL function, range operator, decimal and percentage in

cores in autoscaling policies (noble)

* SOLR-12655: Add Korean morphological analyzer ("nori") to default distribution.

This also adds examples
for configuration in Solr's schema. (Uwe Schindler)

* SOLR-11863: Add knnRegress Stream Evaluator to support nearest neighbor

regression (Joel Bernstein)

* SOLR-12702: Add zscores Stream Evaluator (Joel Bernstein)

* SOLR-12687: Add functions to cache data structures and mathematical models (Joel
Bernstein)

* SOLR-12671: Add robust flag to knnRegress Stream Evaluator (Joel Bernstein)

* SOLR-12660: Add outliers Stream Evaluator to support outlier detection with

probability distributions
(Joel Bernstein)

* SOLR-12634: Add gaussfit Stream Evaluator (Joel Bernstein)

* SOLR-12629: The predict evaluator should work with the polyfit function (Joel
Bernstein)

* SOLR-12715: NodeAddedTrigger should support adding replicas to new nodes by

setting preferredOperation=addreplica.
(shalin)

* SOLR-11861: When creating a configSet via the API, the "baseConfigSet" parameter
now defaults to "_default".
(Amrit Sarkar, David Smiley)

* SOLR-12716: NodeLostTrigger should support deleting replicas from lost nodes by

setting preferredOperation=deletenode.
(shalin)

* SOLR-9418: Added a new (experimental) PhrasesIdentificationComponent for

identifying potential phrases
in query input based on overlapping shingles in the index. (Akash Mehta, Trey
Grainger, hossman)

* SOLR-11943: Add machine learning functions for location data (Joel Bernstein)

* SOLR-12612: Cluster properties restriction of known keys only is relaxed, and now
unknown properties starting with "ext."
will be allowed. This allows custom to plugins set their own cluster properties.
(Jeffery Yuan via Tomás Fernández Löbbe)
* SOLR-12357: Time Routed Aliases now have a preemptiveCreateMath option to
preemptively and asynchronously create the
next collection in advance as new data gets within this time window of the end.
(Gus Heck, David Smiley)

Bug Fixes
----------------------

* SOLR-12449: Response /autoscaling/diagnostics shows improper json (noble)

* SOLR-11676: Keep nrtReplicas and replicationFactor in sync while creating a

collection and modifying a collection
(Varun Thacker)

* SOLR-12489: User specified replicationFactor and maxShardsPerNode is used when

specified during a restore operation.
A user can now specify nrtReplicas/tlogReplicas/pullReplicas while restoring the
collection.
Specifying replicationFactor or nrtReplicas have the same effect and only one can
be specified (Varun Thacker)

* SOLR-11216: Race condition in PeerSync (Cao Manh Dat)

* SOLR-11807: Restoring collection now treats maxShardsPerNode=-1 as unlimited

(Varun Thacker)

* SOLR-12413: If Zookeeper was pre-loaded with data before first-use, then the
aliases information would be ignored.
(David Smiley, Gaël Jourdan, Gus Heck)

* SOLR-12482: Config API returns status 0 for failed operations. (Steve Rowe)

* SOLR-12513: Reproducing TestCodecSupport.testMixedCompressionMode failure (Erick

Erickson)

* SOLR-11665: Improve error handling of shard splitting. Fix splitting of mixed

replica types. (ab)

* SOLR-12326: JSON Facet API: terms facet shard requests now indicate if they have
more buckets to prevent
unnecessary refinement requests. (yonk)

* SOLR-12427: Improve error message for invalid 'start', 'rows' parameters.

(Munendra S N via Jason Gerlowski)

* SOLR-12395: Make 'significantTerms' the SignificantTermsQParserPlugin's name and

deprecate its old 'sigificantTerms' name.
(Tobias Kässmann, Christine Poerschke)

* SOLR-12533 Collection collection fails if metrics are called during core creation
(Peter Cseh, Mano Kovacs)

* SOLR-2834: Fix SolrJ Field and Document analyzes for types that include
CharacterFilter (Alexandre Rafalovitch)

* SOLR-12516: Fix some bugs in 'type:range' Facet refinement when sub-facets are
combined with non
default values for the 'other' and 'include' options. (hossman)
* SOLR-12343: Fixed a bug in JSON Faceting that could cause incorrect counts/stats
when using non default
sort options. This also adds a new configurable "overrefine" option. (Yonik
Seeley, hossman)

* SOLR-12553: Allow SignificantTerms Query Parser to use local parameters

(Alexandre Rafalovitch)

* SOLR-12570: OpenNLPExtractNamedEntitiesUpdateProcessor cannot support multi

fields because pattern replacement
doesn't work correctly. (Koji Sekiguchi)

* SOLR-12576: Update ref guide for additional information displayed in cloud view
(Erick Erickson)

* SOLR-12597: Migrate API should fail requests that do not specify split.key
parameter (shalin)

* SOLR-12477: An update would return a client error(400) if it hit a

AlreadyClosedException.
We now return the error as a server error(500) instead (Jeffery via Varun
Thacker)

* SOLR-12606: Fix InfixSuggestersTest.testShutdownDuringBuild() failures. (Steve

Rowe)

* SOLR-12607: Fixed two separate bugs in shard splits which can cause data loss.
The first case is when using TLOG
replicas only, the updates forwarded from parent shard leader to the sub-shard
leader are written only in tlog and
not the index. If this happens after the buffered updates have been replayed then
the updates can never be executed
even though they remain the transaction log. The second case is when
synchronously forwarding updates to sub-shard
leader fails and the underlying errors are not propagated to the client. (Cao
Manh Dat, shalin)

* SOLR-12344: SolrSlf4jReporter doesn't set MDC context. (ab)

* SOLR-12594: MetricsHistoryHandler.getOverseerLeader fails when hostname contains

hyphen. (ab)

* SOLR-12615: HashQParserPlugin will no longer throw an NPE if the hash key field
is a string when there are documents
with empty values. All documents with empty values ( string , numeric ) will be
processed by worker=0
This would fix the NPE when using the search stream with partitionKeys. (Varun
Thacker)

* SOLR-11770: NPE in tvrh if no field is specified and document doesn't contain any
fields with term vectors

* SOLR-12541: Metrics handler throws an error if there are transient cores. (ab)

* SOLR-12470: Search Rate Trigger multiple bug fixes, improvements and

documentation updates. (ab)

* SOLR-12665: Autoscaling policy not being refreshed due to caching (noble)

* SOLR-12649: CloudSolrClient retries requests unnecessarily exception from server
(noble, shalin)

* SOLR-12670: RecoveryStrategy logs wrong wait time when retrying recovery.

(shalin)

* SOLR-12668: Autoscaling trigger listeners should be executed in the order of

their creation. (ab)

* SOLR-12475: Fix MaxSizeAutoCommitTest failures (Rupa Shankar, Anshum Gupta)

* SOLR-12674: RollupStream should not use the HashQueryParser for 1 worker. (Varun
Thacker)

* SOLR-12679: MiniSolrCloudCluster internal jetty list should never have duplicates

(shalin)

* SOLR-12598: Do not fetch non-stored fields (Nikolay Khitrin, Erick Erickson)

* SOLR-12683: HashQuery will throw an exception if more than 4 partitionKeys is

specified.
Earlier after the 4th partitionKey the keys would be silently ignored. (Varun
Thacker)

* SOLR-10028: Fix and improvements to SegmentsInfoRequestHandlerTest (Christine

Poerschke, Tomás Fernández Löbbe)

* SOLR-11585: Solr SQL does not work with point numeric fields (Joel Bernstein,
Kiran Chitturi)

* SOLR-12704: Guard AddSchemaFieldsUpdateProcessorFactory against null field names

and field values.
(Steve Rowe, Varun Thacker)

* SOLR-12733: SolrMetricReporterTest failure (Erick Erickson, David Smiley)

* SOLR-12765: Incorrect format of JMX cache stats. (Bojan Smid, ab)

* SOLR-12749: timeseries() expression missing sum() results for empty buckets (Joel
Bernstein)

Optimizations
----------------------

* SOLR-12350: Do not use docValues as stored for _str (copy)fields in _default

configset (janhoy)

* SOLR-12455: Refactor JSON serialization code into SolrJ package (noble)

* SOLR-11654: Time Routed Alias will now route documents to the ideal shard of a
collection, thus avoiding a hop.
Usually documents were already routed well but not always. (Gus Heck, David
Smiley)

* SOLR-11598: The export handler does not limit users to 4 sort fields and is now
unlimited. However the speed at
which we can export is directly proportional to the number of sort fields
specified. This change also allows streaming
 expressions to group by on more than 4 fields. (Aroop Ganguly, Amrit Sarkar,
Varun Thacker)

* SOLR-12305: When a replica is applying updates, some kind of updates can skip
buffering for faster recovery.
(Cao Manh Dat)

* SOLR-12509: Improve SplitShardCmd performance and reliability. A new method of

splitting has been
introduced (splitMethod=link) which uses hard-linking of index files when
possible, resulting in
significant speedups and reduced CPU / IO load on shard leader. (ab)

* SOLR-11881: Retry update requests sent by leaders to it's followers (Varun

Thacker, Mark Miller, Tomás Fernández Löbbe)

* SOLR-12616: Optimize Export writer upto 4 sort fields to get better performance.
This was removed in SOLR-11598 but brought back in the same version (Amrit
Sarkar, Varun Thacker)

* SOLR-12572: While exporting documents using the export writer, if a field is

specified as a sort parameter and also
in the fl (field list) parameter, we save on one doc-value lookup. This can bring
performance improvements of 15%
and upwards depending on how many fields are in common. (Amrit Sarkar, Varun
Thacker)

* SOLR-10697: HttpShardHandler now uses a default of 100k as maxConnections (10k

previously) and default
maxConnectionsPerHost as 100k (20 previously). They are now consisent with the
UpdateShardHandler defaults.
(Varun Thacker)

* SOLR-12723: Reduce object creation in HashBasedRouter. (ab)

* SOLR-12766: When retrying internal requests, backoff only once for the full batch
of retries (Tomás Fernández Löbbe)

Other Changes
----------------------

* SOLR-12208: Renamed the autoscaling variable 'INDEX.sizeInBytes' to

'INDEX.sizeInGB' (noble)

* SOLR-12523: Improve error reporting and docs regarding Collection backup feature
shared-fs requirement (janhoy)

* SOLR-12468: Upgrade Jetty to 9.4.11.v20180605 (Michael Braun, shalin)

* SOLR-12527: factor out a test-framework/ConfigRequest class (Christine Poerschke)

* SOLR-12412: Leader should give up leadership when IndexWriter.tragedy occur (Cao

Manh Dat, Tomas Fernandez-Lobbe)

* SOLR-12551: Upgrade to Tika 1.18 (Tim Allison via Erick Erickson)

* SOLR-12464: Reduce Overseer.close() logging (for non-Overseer leaders) (Christine

Poerschke)
* SOLR-12454: Tweak Overseer leadership transition related logging for easier
troubleshooting. (Christine Poerschke)

* SOLR-12574: Put under a common "significantTerms" bucket all output by

SignificantTerms Query Parser (Alexandre Rafalovitch)

* SOLR-12164: Improve Ref Guide main landing page. (Cassandra Targett)

* SOLR-10984: Clean up web.xml, removing old redirects and outdated comments (Varun
Thacker, janhoy)

* SOLR-12617: Remove Commons BeanUtils as a dependency (Varun Thacker)

* SOLR-11008: Use a lighter config for MetricsHandlerTest and ensure the core is up
before the test starts (Varun Thacker)

* SOLR-11766: Move Streaming Expressions section in Ref Guide to be a top-level

section. (Cassandra Targett)

* SOLR-12656: ShardSplitTest should extend AbstractFullDistribZkTestBase instead of

BasicDistributedZkTest. (shalin)

* LUCENE-8456: Upgrade Apache Commons Compress to v1.18 (Steve Rowe)

* SOLR-12014: Cryptic error message when creating a collection with sharding that
violates autoscaling policies (noble)

* SOLR-12680: Fix ClassCastException and AIOOBE in TestSolrConfigHandlerConcurrent.

(shalin)

* SOLR-12675: Make LeaderVoteWaitTimeoutTest more resilient against side effects of

test methods. (shalin)

* SOLR-12130: CdcrReplicationDistributedZkTest is broken into two test classes,

CdcrOpsAndBoundariesTest which does
not require node restarts and CdcrWithNodesRestartsTest which does. The tests
themselves are made faster and more
resilient to spurious failures. (Varun Thacker, Amrit Sarkar via shalin)

* SOLR-12625: Combine SolrDocumentFetcher and RetrieveFieldsOptimizer (Erick

Erickson)

* SOLR-12690: Regularize LoggerFactory declarations (Erick Erickson)

* SOLR-12590: Improve Solr resource loader coverage in the ref guide.

(Steve Rowe, Cassandra Targett, Christine Poerschke)

* SOLR-12744: Improve logging messages and verbosity around recoveries (Cao Manh
Dat, Varun Thacker)

* SOLR-8742: In HdfsDirectoryTest replace RAMDirectory usages with

ByteBuffersDirectory.
(hossman, Mark Miller, Andrzej Bialecki, Steve Rowe)

* SOLR-12771: Improve Autoscaling Policy and Preferences documentation. (hossman,

Steve Rowe)

================== 7.4.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.17
Carrot2 3.16.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.11
Jetty 9.4.10.v20180503

Upgrade Notes
----------------------

* SOLR-7887: Solr now uses Log4J 2.11 . The log4j configuration is now in
log4j2.xml rather than log4j.properties files.
This is a server side change only and clients using SolrJ won't need any changes.
Clients can still use any logging
implementation which is compatible with SLF4J. We now let log4j2 handle rotation
of solr logs at startup, and bin/solr
start scripts will no longer attempt this nor move existing console or GC logs
into logs/archived either (SOLR-12144).

* SOLR-11673: Slave doesn't commit empty index when completely new index is
detected on master during replication.
To return the previous behavior pass false to skipCommitOnLeaderVersionZero in
slave section of replication
handler configuration, or pass it to the fetchindex command.

* SOLR-11453: Configuring slowQueryThresholdMillis now logs slow requests to a

separate file - solr_slow_requests.log.
Previously they would get logged in the solr.log file.

New Features
----------------------

* SOLR-12396: Upgrade Carrot2 to 3.16.0, HPPC to 0.8.1, morfologik to 2.1.5. (Dawid

Weiss)

* SOLR-11200: A new CMS config option 'ioThrottle' to manually enable/disable

ConcurrentMergeSchedule.doAutoIOThrottle. (Amrit Sarkar, Nawab Zada Asad iqbal
via Dawid Weiss)

* SOLR-11670: Implement a periodic house-keeping task. This uses a scheduled

autoscaling trigger and
currently performs cleanup of old inactive shards. (ab, shalin)

* SOLR-12015: Add support "add-distinct" in AtomicURP so that we can use the 'add-
distict' as a request parameter e.g:
atomic.<multival-field-name>=add-distict (Amrit Sarkar via noble)

* SOLR-7887: Upgrade Solr to use Log4J 2.11

(Tim Potter, Keith Laban, Shawn Heisey, Ralph Goers, Erick Erickson, Varun
Thacker)

* SOLR-12139: The "eq" (equals) function query now works with string fields, string
literals, and perhaps anything.
(Andrey Kudryavtsev, David Smiley)
* SOLR-10783: Add support for Hadoop Credential Provider as SSL/TLS store password
source.
(Mano Kovacs via Mark Miller)

* SOLR-12036: Factor out DefaultStreamFactory solrj class. (Christine Poerschke)

* SOLR-12151: Add abstract MultiSolrCloudTestCase class. (Christine Poerschke)

* SOLR-12181: Add index size autoscaling trigger, based on document count or size
in bytes. (ab)

* SOLR-11982: Add possibility to define replica order with the shards.preference

parameter to e.g. prefer PULL replicas
for distributed queries. (Ere Maijala, Tomás Fernández Löbbe)

* SOLR-11336: DocBasedVersionConstraintsProcessorFactory is more extensible and now

supports a list of versioned fields.
(versionField config may now be a comma-delimited list). (Michael Braun via David
Smiley)

* SOLR-11913: SolrJ SolrParams now implements Iterable<Map.Entry<String, String[]>>

and also has a stream() method
using it for convenience. (David Smiley, Tapan Vaishnav)

* SOLR-11924: Added the ability to listen to changes in the set of active

collections in a cloud
in the ZkStateReader, through the CloudCollectionsListener. (Houston Putman,
Dennis Gove)

* SOLR-8998: introducing uniqueBlock(_root_) aggregation as faster alternative to

unique(_root_) for counting
child value facets in parents via json.facet on block index (Dr Oleg Savrasov,
Mikhail Khludnev)

* SOLR-12278: Add IgnoreLargeDocumentProcessFactory (Cao Manh Dat, David Smiley)

* SOLR-11277: Add auto hard-commit settings based on tlog size (Rupa Shankar,
Anshum Gupta)

* SOLR-9480: A new 'relatedness()' aggregate function for JSON Faceting to enable

building Semantic
Knowledge Graphs. (Trey Grainger, hossman)

* SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a separate

file - solr_slow_requests.log.
(Shawn Heisey, Remko Popma, Varun Thacker)

* SOLR-12401: Add getValue() and setValue() Stream Evaluators (Joel Bernstein,

janhoy)

* SOLR-12378: Support missing versionField on indexed docs in

DocBasedVersionConstraintsURP.
(Oliver Bates, Michael Braun via Mark Miller)

* SOLR-12388: Enable a strict ZooKeeper-connected search request mode, in which

search
requests will fail when the coordinating node can't communicate with ZooKeeper,
by setting the "shards.tolerant" param to "requireZkConnected". (Steve Rowe)
* SOLR-9685: #Tagging queries in JSON Query DSL, equivalent to LocalParams based
query/filter
tagging. Multiple tags are comma separated.
LocalParams Example : {!tag=colorfilt}color:blue
Equivalent JSON Example : { "#colorfilt" : "color:blue" }
(Dmitry Tikhonov, Mikhail Khludnev, yonik)

* SOLR-12328: JSON Facet API: Domain change with graph query.

(Daniel Meehl, Kevin Watters, yonik)

* SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a separate

file - solr_slow_requests.log.
(Shawn Heisey, Remko Popma, Varun Thacker)

* SOLR-12401: Add getValue() and setValue() Stream Evaluators (Joel Bernstein,

janhoy)

* SOLR-11779, SOLR-12438: Basic long-term collection of aggregated metrics.

Historical data is
maintained as multi-resolution time series using round-robin databases in the
'.system'
collection. New /admin/metrics/history API allows retrieval of this data in
numeric
or graph formats. (ab)

* SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,

pullReplicas (noble)

* SOLR-12389: support deeply nested json objects in clusterprops.json (noble)

* SOLR-12376: Added the TaggerRequestHandler (AKA SolrTextTagger) for tagging text.

It's used as a component of
NER/ERD systems including query-understanding. See the ref guide for more info.
(David Smiley)

* SOLR-12266: Add discrete Fourier transform Stream Evaluators (Joel Bernstein)

* SOLR-12158: Allow the monteCarlo Stream Evaluator to support variables (Joel

Bernstein)

* SOLR-11734: Add ones and zeros Stream Evaluators (Joel Bernstein)

* SOLR-12273: Create Stream Evaluators for distance measures (Joel Bernstein)

* SOLR-12159: Add memset Stream Evaluator (Joel Bernstein)

* SOLR-12221: Add valueAt Stream Evaluator (Joel Bernstein)

* SOLR-12175: Add random field type and dynamic field to the default managed-schema
(Joel Bernstein)

Bug Fixes
----------------------

* SOLR-5351: Fixed More Like This Handler to use all fields provided in mlt.fl when
used with
content stream. The similarity is calculated between the content stream's value
and all
 fields listed in mlt.fl. (Dawid Weiss)

* SOLR-12103: Raise CryptoKeys.DEFAULT_KEYPAIR_LENGTH from 1024 to 2048. (Mark

Miller)

* SOLR-12107: Fixed a error in [child] transformer that could ocur if documentCache

was not used (hossman)

* SOLR-12108: Fixed the fallback behavior of [raw] and [xml] transformers when an
incompatble 'wt' was
specified, the field value was lost if documentCache was not used. (hossman)

* SOLR-11551: Standardize CoreAdmin API success/failure status codes (Jason

Gerlowski, Steve Rowe)

* SOLR-12035: ExtendedDismaxQParser fails to include charfilters in nostopanalyzer

(Tim Allison via
Tomás Fernández Löbbe)

* SOLR-10734: AtomicUpdateRequestProcessor can cause wrong/old values to be set

under concurrent updates for the same
document. Multithreaded test for AtomicUpdateRequestProcessor was also beefed up
and fixed.
(Ishan Chattopadhyaya, Noble Paul, Amrit Sarkar, shalin)

* SOLR-11673: By default slave doesn't commit empty index when completely new index
appears on master.
See Upgrade Notes to find a way to get back to the previous behavior. (Mikhail
Khludnev)

* SOLR-9399: Delete requests do not send credentials & fails for Basic
Authentication
(Susheel Kumar, Aibao Luo, Nikkolay Martinov via Erick Erickson)

* SOLR-12172: Fixed race condition that could cause an invalid set of collection
properties to be kept in
memory when multiple collection property changes are done in a short period of
time. (Tomás Fernández Löbbe)

* SOLR-11929: UpdateLog metrics are not initialized on core reload. (ab, Steve
Rowe)

* SOLR-11882: SolrMetric registries retained references to SolrCores when closed. A

change of SolrMetricMAnager.registerGauge and
SolrMetricProducer.initializeMetrics
method signatures was required to fix it. Third party components may continue to
use the old API
but should be updated to avoid this bug (Eros Taborelli, Erick Erickson, ab)

* SOLR-12199: TestReplicationHandler.doTestRepeater(): TEST_PORT interpolation

failure:
Server refused connection at: http://127.0.0.1:TEST_PORT/solr (Mikhail Khludnev,
Dawid Weiss, Steve Rowe)

* SOLR-12096: Fixed inconsistent results format of subquery transformer for

distributed search (multi-shard).
(Munendra S N, Mikhail Khludnev via Ishan Chattopadhyaya)

* SOLR-12207: Just rethrowing AssertionError caused by jdk bug in reflection with

invocation details.
(ab, Dawid Weiss, Mikhail Khludnev)

* SOLR-12155: Exception from UnInvertedField constructor puts threads to infinite

wait.
(Andrey Kudryavtsev, Mikhail Khludnev)

* SOLR-12201: TestReplicationHandler.doTestIndexFetchOnLeaderRestart(): handle

unexpected replication failures
(Steve Rowe)

* SOLR-12190: Need to properly escape output in GraphMLResponseWriter. (yonik)

* SOLR-12214: Leader may skip publish itself as ACTIVE when its last published
state is DOWN (Cao Manh Dat)

* SOLR-12150: Fix a test bug in CdcrBidirectionalTest.testBiDir (Steve Rowe, Amrit

Sarkar via Varun Thacker)

* SOLR-10513: ConjunctionSolrSpellChecker did not work with

LuceneLevenshteinDistance (Amrit Sarkar via James Dyer)

* SOLR-11840: Fix bin/solr help-text inconsistencies (Jason Gerlowski)

* SOLR-10169: PeerSync will hit an NPE on no response errors when looking for
fingerprint. (Erick Erickson)

* SOLR-12187: Replica should watch clusterstate and unload itself if its entry is
removed (Cao Manh Dat)

* SOLR-6286: TestReplicationHandler.doTestReplicateAfterCoreReload(): stop checking

for identical
commits before/after master core reload; and make non-nightly mode test 10 docs
instead of 0.
(shalin, hossman, Mark Miller, Steve Rowe)

* SOLR-9304: Fix Solr's HTTP handling to respect '-Dsolr.ssl.checkPeerName=false'

aka SOLR_SSL_CHECK_PEER_NAME
(Shawn Heisey, Carlton Findley, Robby Pond, hossman)

* SOLR-12250: NegativeArraySizeException on TransactionLog if previous document

more than 1.9GB (Cao Manh Dat)

* SOLR-12253: Remove optimize button from the core admin page (Erick Erickson)

* SOLR-11833: Allow searchRate trigger to delete replicas. Improve configurability

of the trigger by specifying
upper / lower thresholds and respective actions (ab)

* SOLR-12261: If you attempt to delete a collection immediately after deleting an

alias it used to be a part of, you may
get an error that's it's a member of that alias. This check now ensures the
alias state is sync()'ed with ZK first.
(David Smiley)

* SOLR-12275: wrong caching for {!filters} as well as for `filters` local param in
{!parent} and {!child}
(David Smiley, Mikhail Khluldnev)
* SOLR-12284: WordBreakSolrSpellchecker will no longer add parenthesis in
collations when breaking words in
non-boolean queries. (James Dyer)

* SOLR-12290: Do not close any servlet streams and improve our servlet stream
closing prevention code for users
and devs. (Mark Miller, janhoy, Andrzej Bialecki)

* SOLR-12293: Updates need to use their own connection pool to maintain connection
reuse and prevent spurious
recoveries. (Mark Miller)

* SOLR-12308: LISTALIASES is now assured to return an up-to-date response. Also,

the test utility
MiniSolrCloudCluster.deleteAllCollections will now first delete aliases since a
collection cannot be deleted if an
alias refers to it. (David Smiley)

* SOLR-12192: Fixed a solr cli error message when ulimits are unlimited. (Martijn
Koster)

* SOLR-12258: A V2 request referencing a collection or alias may fail to resolve it

if it was just recently created.
Now we sync with ZooKeeper and try one more time. V1 partially did this but only
for aliases; now it does both.
(David Smiley)

* SOLR-12170: JSON Facet API: Terms facet on a date field sometimes failed with an
exception complaining
about "Invalid Date String". (yonik)

* SOLR-12307: exiting OverseerTriggerThread without endless noise in log when

Zookeeper session is expired
(Mikhail Khludnev)

* SOLR-12200: abandon OverseerExitThread when ZkController is closed. (Mikhail

Khludnev)

* SOLR-12355: Fixes hash conflict in HashJoinStream and OuterHashJoinStream (Dennis

Gove)

* SOLR-12377: Don't spin off overseer when Zk controller is closed (Mikhail

Khludnev)

* SOLR-3567: Spellcheck custom parameters not being passed through due to wrong
prefix creation.
(Josh Lucas via shalin)

* SOLR-12358: Autoscaling suggestions fail randomly with sorting (noble)

* SOLR-12294: update processors loaded from runtime jars fail to load if they are
specified
in an update processor chain (noble)

* SOLR-12314: Use http timeout's defined in solr.xml for creating

ConcurrentUpdateSolrClient during
indexing requests between leader and replica ( Mark Miller, Varun Thacker)

* SOLR-12290: Do not close any servlet streams and improve our servlet stream
closing prevention code for users
and devs. (Mark Miller)

* SOLR-12374: SnapShooter.getIndexCommit can forget to decref the searcher; though

it's not clear in practice when.
(David Smiley)

* SOLR-12417: velocity response writer should enforce valid function name for
v.json parameter (Mano Kovacs, yonik)

* SOLR-12271: Fixed bug in how Analytics component reads negative values from float
and double fields. (Houston Putman)

* SOLR-12433: Recovering flag of a replica is set equals to leader even it failed

to receive update
on recovering. (Cao Manh Dat)

* SOLR-12354: Register the /admin/info/key end-point at the startup time to avoid

404 (noble)

* SOLR-12445: Upgrade Dropwizard Metrics to version 3.2.6. (ab)

* SOLR-12434: bin/solr {config,healthcheck} ignore ZK_HOST in solr.in.{sh,cmd}

(Steve Rowe)

* SOLR-12481: update.autoCreateFields must be set via Config API command 'set-user-

property',
but 'bin/solr create' tells users to use the default action 'set-property', which
fails
because the property is not editable. (Steve Rowe)

* SOLR-12416: When creating a time routed alias, the router.autoDeleteAge option

wasn't considered.
(Joachim Sauer via David Smiley)

* SOLR-12450: Don't allow referal to external resources in various config files

(CVE-2018-8026).
(Yuyang Xiao, Uwe Schindler)

Optimizations
----------------------

* SOLR-11920: IndexFetcher now fetches only those files (from master/leader) that
are different. This
differential fetching now speeds up recovery times when full index replication is
needed, but only
a few segments diverge. (Ishan Chattopadhyaya, Shaun Sabo, John Gallagher)

* SOLR-11731: LatLonPointSpatialField can now decode points from docValues when

stored=false docValues=true,
albeit with maximum precision of 1.37cm (Karthik Ramachandran, David Smiley)

* SOLR-11891: DocStreamer now respects the ReturnFields when populating a

SolrDocument, reducing the
number of unneccessary fields a ResponseWriter will see if documentCache is used
(wei wang, hossman)

* SOLR-12312: Replication IndexFetcher should cap its internal buffer size when the
file being transferred is small.
 (Jeff Miller, David Smiley)

* SOLR-12333: Removed redundant lines for handling lists in JSON reponse writers.
(David Smiley via Mikhail Khludnev)

* SOLR-11880: Avoid creating new exceptions for every request made to

MDCAwareThreadPoolExecutor by distributed
search and update operations. (Varun Thacker, shalin)

* SOLR-12375: Optimize Lucene ScoreMode use:

A non-cached filter query could be told incorrectly that scores were needed.
The /export (ExportQParserPlugin) would declare incorrectly that scores are
needed.
Expanded docs (expand component) could be told incorrectly that scores are
needed. (David Smiley)

* SOLR-12338: Replay buffering tlog in parallel. (Cao Manh Dat, David Smiley)

* SOLR-12366: A slow "live docs" implementation was being used instead of a bitset.
Affects classic faceting
enum method, JSON Facets enum method, UnInvertedField faceting,
GraphTermsQParser, JoinQParser. (David Smiley)

* SOLR-12337: Remove the obsolete QueryWrapperFilter intermediate wrapper, which

also removed some needless uses of
SolrConstantScoreQuery as well. QWF since v5.4.0 sometimes needlessly internally
executed and cached the query.
Affects ExpandComponent, ChildDocTransformer, CurrencyFieldType, TermsQParser.
(David Smiley)

* SOLR-9922: Write buffering updates to another tlog. (Cao Manh Dat)

* SOLR-12233: QParserPlugin's built-in static registry now holds actual

QParserPlugin instances instead of class
references. This is consistent with other plugin registries and allows a
SolrCore to load faster.
(Jeff Miller, David Smiley)

* SOLR-12198: Stream Evaluators should not copy matrices needlessly (Joel

Bernstein)

Other Changes
----------------------

* SOLR-12018: Remove comments.apache.org integration for the Ref Guide; the

comments system has been down since December 2017 and there is no concrete plan to
bring it back. (Cassandra Targett)

* SOLR-12076: Remove unnecessary printLayout usage in CDCR tests (Varun Thacker)

* SOLR-12086: Fix format problem in FastLRUCache description string shown on Cache

Statistics page.
(Sathiya N Sundararajan via shalin)

* SOLR-12090: Move DistribStateManager, NodeStateProvider and SolrCloudManager

interfaces out of the
autoscaling package. (shalin)

* SOLR-12091: Rename TimeSource.getTime to getTimeNs. (ab)

* SOLR-12101: ZkTestServer was not handling connection timeout settings properly.
(Gus Heck via Mark Miller)

* SOLR-11331: Ability to run and debug standalone Solr and a single node SolrCloud
server from Eclipse.
Also being able to run all Lucene and Solr tests as a configuration (Karthik
Ramachandran via
Varun Thacker, Uwe Schindler)

* SOLR-12118: Solr Ref-Guide can now use some ivy version props directly as
attributes in content (hossman)

* SOLR-12152: Split up TriggerIntegrationTest into multiple tests to isolate and

increase reliability. (shalin)

* SOLR-12133: Fix race conditions that caused TriggerIntegrationTest.testEventQueue

and testNodeMarkersRegistration
to fail. (Mark Miller, shalin)

* SOLR-12169: Fix ComputePlanActionTest.testSelectedCollections fails on jenkins by

aggressively cleaning up
trigger state left by other test methods in the test setup. (shalin)

* SOLR-12144: SOLR_LOG_PRESTART_ROTATION now defaults to false, we leverage log4j2

for log rotation on startup (janhoy)

* SOLR-12095: AutoScalingHandler validates trigger configurations before updating

Zookeeper. (ab)

* SOLR-12165: Ref Guide: DisMax default mm param value is improperly documented as

100%. (Steve Rowe)

* SOLR-12154: Disallow explicit usage of Log4j2 logger via forbidden APIs. (Varun
Thacker, Tomás Fernández Löbbe)

* SOLR-12176: Improve FORCELEADER to handle the case when a replica win the
election but does not present
in clusterstate (Cao Manh Dat)

* SOLR-12134: ref-guide 'bare-bones html' validation is now part of 'ant

documentation' and validates
javadoc links locally. (hossman)

* SOLR-12142: EmbeddedSolrServer should use req.getContentWriter (noble)

* SOLR-12252: Fix minor compiler and intellij warnings in autoscaling policy

framework. (shalin)

* SOLR-11914: The following SolrParams methods are now deprecated: toSolrParams

(use NamedList.toSolrParams instead),
toMap, toMultiMap, toFilteredSolrParams, getAll. The latter ones have no direct
replacement but are easy to
implement yourself as-needed. (David Smiley)

* SOLR-10036: Upgrade jackson from 2.5.4 to 2.9.5 (Shashank Pedamallu, Shawn

Heisey, Varun Thacker)

* SOLR-12289: Add more MDC logging information to collection admin requests (Varun
Thacker)

* SOLR-12288: Add more MDC logging information to core admin requests (Varun
Thacker)

* SOLR-12265: Upgrade Jetty to 9.4.10.v20180503 (Varun Thacker, Steve Rowe)

* SOLR-12374: Added SolrCore.withSearcher(lambda) to make grabbing the searcher

easier than the more awkward
RefCounted API. (David Smiley)

* SOLR-12183: Refactor Streaming Expression test cases (Joel Bernstein)

* SOLR-12437: Document 'bin/solr config' in the ref guide. (Steve Rowe)

* SOLR-12435: Fix bin/solr help and ref guide text to describe ZK_HOST in
solr.in.sh/solr.in.cmd
as an alternative to -z cmdline param. (Steve Rowe)

* SOLR-12428: Solr LTR jar now included in _default configset's solrconfig.xml

(Ishan Chattopadhyaya)

================== 7.3.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.17
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.11
Jetty 9.4.8.v20171121

Bug Fixes
----------------------

* SOLR-12204: Upgrade commons-fileupload dependency to 1.3.3 to address CVE-2016-

1000031. (Steve Rowe)

* SOLR-12256: Fixed some eventual-consistency issues with collection aliases by

using ZooKeeper.sync(). (David Smiley)

* SOLR-12087: Deleting replicas sometimes fails and causes the replicas to exist in
the down
state (Cao Manh Dat)

* SOLR-12146: LIR should skip deleted replicas (Cao Manh Dat)

* SOLR-12066: Cleanup deleted core when node start (Cao Manh Dat)

* SOLR-12065: A successful restore collection should mark the shard state as active
and not buffering
(Rohit, Varun Thacker)

* SOLR-11724: Cdcr bootstrapping should ensure that non-leader replicas should sync
with the leader
 (Amrit Sarkar, Varun Thacker)

* SOLR-12202: Fix errors in solr-exporter.cmd. (Minoru Osuka via koji)

* SOLR-12316: Do not allow to use absolute URIs for including other files in
solrconfig.xml
and schema parsing (CVE-2018-8010). (Ananthesh, Ishan Chattopadhyaya, Uwe
Schindler)

================== 7.3.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.16
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.11
Jetty 9.4.8.v20171121

Upgrade Notes
----------------------

* SOLR-11748: The throttling mechanism used to limit the rate of autoscaling events
processed
has been removed. This deprecates the 'actionThrottlePeriodSeconds' setting in
the set-properties
Autoscaling API which is now a no-op. Use the 'triggerCooldownPeriodSeconds'
instead to pause event
processing.

* SOLR-11798: The top-level <highlighting> syntax in solrconfig.xml is now formally

deprecated in favour of <searchComponent> equivalent syntax. See also SOLR-1696.

* SOLR-11809: QueryComponent's rq parameter parsing no longer considers the defType

parameter.

* SOLR-11747: The behaviour of the autoscaling system has been modified to pause
all triggers from execution between
the start of actions and end of cool down period. The triggers will be resumed
after the cool down period expires.
Previously, the cool down period was a fixed period started after actions for a
trigger event complete and during
this time, all triggers continued to run but any events were rejected to be tried
later.

* SOLR-11624: Collections created without specifying a configset name use a copy of

the _default configset since 7.0.
Before 7.3, the copied over configset was named the same as the collection name,
but 7.3 onwards it will be named
with an additional ".AUTOCREATED" suffix.

* SOLR-11702: The old Leader-In-Recovery implementation (SOLR-5495) is now

deprecated and replaced. Solr will support
rolling upgrades from old 7.x versions of Solr to future 7.x releases until the
last release of
 the 7.x major version. This means in order to upgrade to Solr 8 in the future,
you must be on Solr 7.3 or higher.

* LUCENE-8161: If you are using the spatial JTS library with Solr, you must upgrade
to 1.15.0. This new version
of JTS is now dual-licensed to include a BSD style license.

* SOLR-12051: A new mechanism is introduced in SOLR-11702 to maintain consistency

in SolrCloud between leader and replicas.
This mechanism lets Solr know whether a replica is in-sync with the leader or
not, even when the leader is not live.
If all the replicas who participate in the leader election are out-of-sync with
previous leader, the election will
pause until a timeout (named "leaderVoteWait") before allowing an out-of-sync
replica to become leader. Note that the
new leader still needs to contains more updates than any other active replicas in
the same shard. Therefore by
increasing leaderVoteWait will increase the consistency (over availability) of
the system. The default value of
leaderVoteWait is 180,000 ms (3 minutes) and it can be adjusted in the
"solrcloud" section of the solr.xml

* SOLR-11957: The default Solr log file size and number of backups is raised to
32MB and 10 respectively

* SOLR-12067: The default value of `autoReplicaFailoverWaitAfterExpiration` has

been increased to 120 seconds
from the earlier default of 30 seconds. This affects how soon Solr adds new
replicas to replace the replicas
on nodes which have either crashed or shutdown.

New Features
----------------------

* SOLR-11285: Simulation framework for autoscaling. (ab)

* LUCENE-2899: In the Solr analysis-extras contrib, added support for the

OpenNLP-based analysis components in the Lucene analysis/opennlp module:
tokenization, part-of-speech tagging, phrase chunking, and lemmatization.
Also added OpenNLP-based named entity extraction as a Solr update request
processor. (Lance Norskog, Grant Ingersoll, Joern Kottmann, Em, Kai Gülzau,
Rene Nederhand, Robert Muir, Steven Bower, Steve Rowe)

* SOLR-11201: Implement autoscaling trigger for arbitrary metrics that creates

events when
a given metric breaches a threshold (shalin)

* SOLR-11653: TimeRoutedAlias URP now auto-creates new collections on the fly

according to alias metadata
rules that sets the time interval for each collection. An internal Overseer
command "ROUTEDALIAS_CREATECOLL"
was created to facilitate this. (David Smiley)

* SOLR-11062: new tag "diskType" in autoscaling policy (noble)

* SOLR-11063: Suggesters should accept required freedisk as a hint (noble)

* SOLR-3218: Added range faceting support for CurrencyFieldType. This includes

both "facet.range" as well
 as json.facet's "type:range" (Andrew Morrison, Jan Høydahl, Vitaliy Zhovtyuk,
hossman)

* SOLR-11064: Collection APIs should use the disk space hint when using policy
framework (noble)

* SOLR-11854: multivalued primitive fields can now be sorted by implicitly choosing

the min/max
value for asc/desc sort orders. (hossman)

* SOLR-11592: Add OpenNLP language detection to the langid contrib. (Koji, Steve
Rowe)

* SOLR-11648: A new admin UI to display and execute suggestions (Apoorv Bhawsar ,

noble)

* SOLR-11722: Added "time routed alias" creation support to the CREATEALIAS

command. It's for managing multiple
collections partitioned by time. (Gus Heck, David Smiley)

* SOLR-11782: Refactor LatchWatcher.await to protect against spurious wakeup (Tomás

Fernández Löbbe, David Smiley, Dawid Weiss)

* SOLR-11617: Alias properties (formerly "metadata") are now mutable via a new
ALIASPROP command.
These properties are returned from LISTALIASES. (Gus Heck via David Smiley)

* SOLR-11702: Redesign current LIR implementation (Cao Manh Dat, shalin)

* SOLR-11376: Support computing plans for only specific collections. (ab)

* SOLR-11681: Add ttest and pairedTtest Stream Evaluators (Joel Bernstein)

* SOLR-11785: Add multiVariateNormalDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11789: Add integrate Stream Evaluator to support integral calculations (Joel

Bernstein)

* SOLR-11791: Add density Stream Evaluator (Joel Bernstein)

* SOLR-11172: Add Mann-Whitney U test Stream Evaluator (Mathew Skaria, Joel

Bernstein)

* SOLR-11808: Add sumSq Stream Evaluator (Joel Bernstein)

* SOLR-11430: Add lerp and akima Stream Evaluators to support linear and akima
spline interpolation (Joel Bernstein)

* SOLR-11683: Add chiSquareDataSet Stream Evaluator (Joel Bernstein)

* SOLR-10716: Add termVectors Stream Evaluator (Joel Bernstein)

* SOLR-11737: Add kmeans Stream Evaluator to support kmeans clustering (Joel

Bernstein)

* SOLR-11736: Rename knn Streaming Expression to knnSearch and add new knn Stream
Evaluator (Joel Bernstein)

* SOLR-11867: Add indexOf, rowCount and columnCount StreamEvaluators (Joel

Bernstein)

* SOLR-11862: Add fuzzyKmeans Stream Evaluatory (Joel Bernstein)

* SOLR-11890: Add multiKmeans Stream Evaluator (Joel Bernstein)

* SOLR-11916: new SortableTextField which supports analysis/searching just like

TextField, but also
sorting/faceting just like StrField. By default uses only the first 1024 chars
of the original
input string values, but this is configurable. (hossman)

* SOLR-11778: Add per-stage RequestHandler metrics. (ab)

* SOLR-11925: Time Routed Aliases can have their oldest collections automatically
deleted via the "router.autoDeleteAge"
setting. (David Smiley)

* SOLR-11941: Add abstract contrib/ltr AdapterModel to facilitate the development

of scoring models that delegate
scoring to an opaque pre-trained model. (Christine Poerschke)

* SOLR-11689: Add l1norm, l2norm and linfnorm Stream Evaluators (Joel Bernstein)

* SOLR-11588: Add matrixMult Stream Evaluator to support matrix multiplication

(Joel Bernstein)

* SOLR-12006: Add a '*_t' and '*_t_sort' dynamic field for single valued text
fields (Varun Thacker)

* SOLR-11597: Add contrib/ltr NeuralNetworkModel class. (Michael A. Alcorn, Yuki

Yano, Christine Poerschke)

* SOLR-11066: Implement a scheduled autoscaling trigger that runs on a fixed

interval beginning with a
given start time. (David Smiley, Gus Heck, ab, shalin)

* SOLR-11795: Add Solr metrics exporter for Prometheus (Minoru Osuka via koji)

* SOLR-11267: Add support for "add-distinct" atomic update operation (Amrit Sarkar
via noble )

* SOLR-11960: Add collection level properties similar to cluster properties (Peter

Rusko, Tomás Fernández Löbbe)

* SOLR-12077: Add support for autoAddReplicas in the collection creation dialog in

Admin UI. (shalin)

* SOLR-9510: introducing {!filters param=$fq excludeTags=f} query parser.

Introducing {!.. filters=$fq excludeTags=t,q} in {!parent} and {!child} (Dr. Oleg
Savrasov via Mikhail Khludnev)

Bug Fixes
----------------------

* SOLR-11783: Rename core in solr standalone mode is not persisted (Erick Erickson)

* SOLR-11555: If the query terms reduce to nothing, filter(clause) produces an NPE

whereas
 fq=clause does not (Erick Erickson)

* SOLR-11824: Fixed bucket ordering in distributed json.facet type:range when

mincount>0 (hossman)

* SOLR-11821: ConcurrentModificationException in SimSolrCloudTestCase.tearDown

(shalin)

* SOLR-11631: The Schema API should return non-zero status when there are failures.
(Noble Paul, Steve Rowe)

* SOLR-11839: Fix test failures resulting from SOLR-11218 (Erick Erickson)

* SOLR-11794: PULL replicas stop replicating after collection RELOAD (Samuel

Tatipamula, Tomás Fernández Löbbe)

* SOLR-11714: AddReplicaSuggester / ComputePlanAction infinite loop. (ab)

* SOLR-11895: Logging Screen in the Admin UI will now show "No Events available"
when there are no events to show.
Also, the spinner icon is removed to prevent users thinking the page has hung.
(Cassandra Targett, Shawn Heisey)

* SOLR-10525: Stacked recovery requests do no cancel an in progress recovery first.

(Mike Drob via Cao Manh Dat)

* SOLR-11873: Use time based expiration cache in all necessary places in

HdfsDirectoryFactory. (Mihaly Toth via Mark Miller)

* SOLR-11661: New HDFS collection reuses unremoved data from a deleted HDFS
collection with same name causes
inconsistent view of documents (Cao Manh Dat, shalin)

* SOLR-11459: In-place update of nonexistent doc following existing doc update

fails to create the doc.
(Andrey Kudryavtsev via Mikhail Khludnev)

* SOLR-11931: Fix contrib/ltr custom inner class feature/normaliser/model

persistence. (Christine Poerschke)

* SOLR-10261: In case of in-place updates, failure in leader to follower replica

update request now throws the
follower replica in leader-initiated-recovery (Ishan Chattopadhyaya, Steve Rowe)

* SOLR-11898: ConcurrentModificationException when calling

org.apache.solr.core.SolrInfoBean.getMetricsSnapshot
(Jeff Miller via Erick Erickson)

* SOLR-11950: Allow CLUSTERSTATUS "shard" parameter to accept comma (,) delimited

list (Chris Ulicny via Jason Gerlowski)

* SOLR-11739: Fix race condition that made Solr accept duplicate async IDs in
collection API operations (Tomás Fernánadez Löbbe)

* SOLR-11988: Fix exists() method in EphemeralDirectoryFactory/MockDirectoryFactory

to prevent false positives (hossman)

* SOLR-11971: Don't allow referal to external resources in DataImportHandler's

dataConfig request parameter (CVE-2018-1308).
 (麦 香浓郁, Uwe Schindler)

* SOLR-12021: Fixed a bug in ApiSpec and other JSON resource loading that was
causing unclosed file handles (hossman)

* SOLR-10720: Aggressive removal of a collection breaks cluster status API. (Alexey

Serba, shalin)

* SOLR-12050: UTILIZENODE does not enforce policy rules (hossman, noble)

* SOLR-11843: Admin UI collection creation was not properly handling the

router.field and router.name parameters.
Also changed the display label in the Admin UI from routerField to router.field
to match the actual API.
(Shawn Heisey via Cassandra Targett)

* SOLR-12011: Consistence problem when in-sync replicas are DOWN. (Cao Manh Dat)

* SOLR-12020: JSON Facet API: terms facet on date field fails in refinement phase
with
"Invalid Date String" error. (yonik)

* SOLR-12061: Fix substitution bug in API V1 to V2 migration when using SolrJ with
V2 API. (Tomás Fernánadez Löbbe)

* SOLR-11976: TokenizerChain.normalize: only the first filter that is a

MultiTermAwareComponent was participating
in normalization instead of all. This bug normally doesn't matter since
TextField doesn't call it.
(Tim Allison via David Smiley)

* SOLR-12072: Invalid path string using ZkConfigManager.copyConfigDir(String

fromConfig, String toConfig)
(Alessandro Hoss via Erick Erickson)

* SOLR-12064: JSON Facet API: fix bug where a limit of -1 in conjunction with
multiple facets or
missing=true caused an NPE or AIOOBE. (Karthik Ramachandran via yonik)

* SOLR-12083: Fix RealTime GET to work on a cluster running CDCR when using Solr's
in-place updates
(Amrit Sarkar, Varun Thacker)

* SOLR-12063: Fix PeerSync, Leader Election failures and CDCR checkpoint

inconsistencies on a cluster running CDCR
(Amrit Sarkar, Varun Thacker)

* SOLR-12110: Replica which failed to register in Zk can become leader (Cao Manh
Dat)

* SOLR-12129: After the core is reloaded, term of the core will not be watched (Cao
Manh Dat)

* SOLR-12141: Fix "bin/solr" shell scripts (Windows/Linux/Mac) to correctly detect

major Java version
and use numerical version comparison to enforce minimum requirements. Also remove
obsolete "UseParNewGC" option.
This allows to start Solr with Java 10 or later. (Uwe Schindler)
Optimizations
----------------------

* SOLR-11711: Fixed distributed processing of facet.field/facet.pivot sub requests

to prevent requesting
unneccessary and excessive '0' count terms from each shard (Houston Putman via
hossman)

* LUCENE-8149: NRTCachingDirectory does not need to preemptively delete segment

files and generate exceptions
(Erick Erickson)

* SOLR-11879: avoid EOFException for empty input streams (noble)

* SOLR-8327: Cluster state caching to avoid live fetch from ZK of cluster state for
forwarding requests on nodes
that do not host any replica of the collection (Jessica Cheng Mallet, John
Gallagher, Noble Paul, Ishan Chattopadhyaya)

* SOLR-11769: Optimize when useFilterForSortedQuery=true and there are no filter

queries by avoiding
needless match-all-docs bitset construction. (Betim Deva, David Smiley)

Other Changes
----------------------

* SOLR-11629: Add more intuitive CloudSolrClient.Builder constructors (Varun

Thacker, Jason Gerlowski)

* SOLR-11575: Improve ref-guide solrj snippets via mock 'print()' method (Jason
Gerlowski via hossman)

* SOLR-11757: In tests, fix race condition on SolrException.ignoreException.

Also ensure we register "ignore_exception" in @BeforeClass (previously only
@AfterClass) (David Smiley)

* SOLR-11754: Remove AbstractSolrTestCase which has long been supplanted by

SolrTestCaseJ4. (David Smiley)

* SOLR-11701: Upgrade to Tika 1.17 when available (Tim Allison, Karthik

Ramachandran via Erick Erickson)

* SOLR-11703: Solr Should Send Log Notifications if Ulimits are too low (Kevin
Cowan via Erick Eickson)

* SOLR-11793: Reduce code duplication w.r.t. RestTestHarness(es). (Christine

Poerschke)

* SOLR-7733: remove "optimize" from the UI. (Erick Erickson)

* SOLR-11748: Remove Autoscaling action throttle. (shalin)

* SOLR-11805: SolrJ's SolrResponse.getElaspedTime was sometimes a millisecond off.

(David Smiley)

* SOLR-11798: Formally deprecate top-level <highlighting> syntax in solrconfig.xml

in favour of <searchComponent> equivalent syntax. (Christine Poerschke)

* SOLR-11801: Support customisation of the "highlighting" query response element.

 (Ramsey Haddad, Pranav Murugappan, David Smiley, Christine Poerschke)

* SOLR-11692: SolrDispatchFilter's use of a "close shield" in tests should not be

applied to
further servlet chain processing. (Jeff Miller, David Smiley)

* SOLR-11218: Fail and return an error when attempting to delete a collection

that's part of an alias (Erick Erickson)

* SOLR-11817: Move Collections API classes to it's own package (Varun Thacker)

* SOLR-11810: Upgrade Jetty to 9.4.8.v20171121 (Varun Thacker, Erick Erickson)

* SOLR-11747: Pause triggers until actions finish executing and the cool down
period expires. (shalin)

* SOLR-11871: MoveReplicaSuggester should not suggest leader if other replicas are

available (noble)

* SOLR-11624: Collections created from _default configset will now be associated

with a configset with a suffix
.AUTOCREATED. For example, a new collection "mycollection", created without
specifying a configset name, will
use the _default configset and the associated configset name will be
"mycollection.AUTOCREATED". If this
collection is deleted and re-created, the autocreated configset will be left
behind and will be re-used for
the re-created collection (Ishan Chattopadhyaya, Abhishek Kumar Singh)

* SOLR-11051: Use disk free metric in default cluster preferences (noble)

* SOLR-11658: Upgrade ZooKeeper dependency to 3.4.11 (Jason Gerlowski, Steve Rowe,

Erick Erickson)

* SOLR-11480: Remove unused "Admin Extra" files and mentions. (Eric Pugh, Christine
Poerschke)

* SOLR-11067: REPLACENODE should identify appropriate nodes if targetNode is not

provided (noble)

* SOLR-11848: Update Ref Guide to include info on grouping operations and using
curl for large files. (Dariusz Wojtas
via Cassandra Targett)

* SOLR-11613: Make message for missing dataimport config in UI more explicit.

(Shawn Heisey, Amrit Sarkar via
Cassandra Targett)

* SOLR-11933: Make DIH UI page safer by not default checking the clean checkbox
(Eric Pugh via Tomás Fernández Löbbe)

* SOLR-11349: Rename ResponseBuilder's getQueryCommand to createQueryCommand.

(Christine Poerschke)

* SOLR-11902: Clarify in bin/solr help text whether commands can be run remotely
(Jason Gerlowski)

* SOLR-3089: RequestBuilder now exposes isDistrib() method. Using this, plugins can
know whether a request is
 distributed. (Rok Rejc, Frank Wesemann, Abhishek Kumar Singh, Mikhail Khludnev
via Ishan Chattopadhyaya)

* SOLR-8090: Make text elements in the UI darker for better contrast and
readability (Cassandra Targett)

* SOLR-6057: Change highlight color in UI Analysis screen (Cassandra Targett)

* SOLR-11897: Add toggle to disable auto-refresh of Logging page in the UI (Tommy

Marshment-Howell via
Cassandra Targett)

* SOLR-12017: Remove BadApple and AwaitsFix annotations that link to closed JIRAs
(Erick Erickson)

* SOLR-12027: Increase thread lingering timeout to 80s. (Mikhail Khludnev)

* SOLR-10809: Get precommit lint warnings out of Solr core (Erick Erickson)

* SOLR-12028: BadApple and AwaitsFix annotations usage (Erick Erickson, Uwe

Schindler)

* SOLR-12031: Refactor Policy framework to make simulated changes affect more than
a single node (noble)

* SOLR-11957: Increase MaxFileSize=32MB and MaxBackupIndex=10 for

RollingFileAppender in log4j.properties
(Varun Thacker, shalin)

* SOLR-12047: Increase checkStateInZk timeout (Cao Manh Dat, Varun Thacker)

* SOLR-12051: Election timeout when no replicas are qualified to become leader (Cao
Manh Dat)

* SOLR-12067: Increase autoAddReplicas default 30 second wait time to 120 seconds.

(Varun Thacker, Mark Miller via shalin)

* SOLR-12078: Fixed reproducable Failure in

TestReplicationHandler.doTestIndexFetchOnLeaderRestart that happened
due to using stale http connections. (Gus Heck, shalin)

* SOLR-12099: Remove reopenReaders attribute from 'IndexConfig in SolrConfig' page

in ref guide. (shalin)

* SOLR-12098: Document the Lucene spins auto-detection and its effect on CMS
dynamic defaults.
(Cassandra Targett, shalin)

* SOLR-12097: Document the diskType policy attribute and usage of disk space in
Collection APIs.
(Cassandra Targett, shalin)

================== 7.2.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.16
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.20.v20170531

Bug Fixes
----------------------

* SOLR-11771: Overseer can never process some last messages (Cao Manh Dat)

* SOLR-11783: Rename core in solr standalone mode is not persisted (Erick Erickson)

* SOLR-11809: QueryComponent.prepare rq parsing could fail under SOLR 7.2.0 - fix:

QueryComponent's rq parameter parsing no longer considers the defType parameter.
(Christine Poerschke and David Smiley in response to bug report/analysis
from Dariusz Wojtas and Diego Ceccarelli)

* SOLR-11555: If the query terms reduce to nothing, filter(clause) produces an NPE

whereas
fq=clause does not (Erick Erickson)

================== 7.2.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.16
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.20.v20170531

Upgrade Notes
----------------------

* SOLR-11501: Starting a query string with local-params {!myparser ...} is used to

switch the query parser to another,
and is intended for use by Solr system developers, not end users doing searches.
To reduce negative side-effects of
unintended hack-ability, we've limited the cases that local-params will be parsed
to only contexts in which the
default parser is "lucene" or "func".
So if defType=edismax then q={!myparser ...} won't work. In that example, put
the desired query parser into defType.
Another example is if deftype=edismax then hl.q={!myparser ...} won't work for
the same reason. In that example,
either put the desired query parser into hl.qparser or set hl.qparser=lucene.
Most users won't run into these cases
but some will and must change.
If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or
something earlier. (David Smiley)

* SOLR-11501: The edismax parser by default no longer allows subqueries that

specify a Solr parser using either
 local-params, or the older _query_ magic field trick. For example
{!prefix f=myfield v=enterp} or _query_:"{!prefix f=myfield v=enterp}" are
not supported by default anymore.
If you want to allow power-users to do this, set uf=* _query_ or some other value
that includes _query_.
If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or
something earlier. (David Smiley)

New Features
----------------------
* SOLR-11448: Implement an option in collection commands to wait for final results.
(ab)

* SOLR-11072: Implement trigger for searchRate event type. (ab)

* SOLR-11524: A new autoscaling/suggestions API end-point which gives autoscaling

suggestions (noble)

* SOLR-11519: Implement autoscaling suggestions for replica count violations

(noble)

* SOLR-11518: Implement autoscaling Suggestions for freedisk violations (noble)

* SOLR-10132: A new optional facet.matches parameter to return facet buckets only

for terms that match a regular expression. (Gus Heck, Christine Poerschke)

* SOLR-11520: Implement autoscaling suggestions for cores count violations (noble)

* SOLR-11438: Solr should return rf when min_rf is specified for deletes as well as
adds (Erick Erickson)

* SOLR-11003: Support bi-directional syncing of cdcr clusters. We still only

support active indexing in one cluster,
but have the ability to switch indexing clusters and cdcr will replicate
correctly. (Amrit Sarkar, Varun Thacker)

* SOLR-11538: Implement suggestions for port,ip_*, nodeRole,sysprop.*, metrics:*

(noble)

* SOLR-11487: Collection Aliases may now have metadata (currently an internal

feature).
(Gus Heck, David Smiley)

* SOLR-11542: New TimeRoutedAliasUpdateProcessor URP that routes documents to

another collection
in the same Alias defined set based on a time field (currently an internal
feature).
(David Smiley)

* SOLR-9743: A new UTILIZENODE command (noble)

* SOLR-11202: Implement a set-property command for AutoScaling API. (ab, shalin)

* SOLR-11250: A new DefaultWrapperModel class for loading of large and/or

externally stored
LTRScoringModel definitions. (Yuki Yano, shalin, Christine Poerschke)

* SOLR-11662: New synonymQueryStyle option to configure whether SynonymQuery, a

DisjunctionMaxQuery, or BooleanQuery
 occurs over query terms that overlap their position. (Doug Turnbull, David
Smiley)

* SOLR-11429: Add loess Stream Evaluator to support local regression (Joel

Bernstein)

* SOLR-11568: Add matrix Stream Evaluator to support efficient matrix operations

(Joel Bernstein)

* SOLR-11566: Add transpose Stream Evaluator to support transposing of matrices

(Joel Bernstein)

* SOLR-11565: Add unit Stream Evaluator to support unitizing of vectors (Joel

Bernstein)

* SOLR-11567: Add triangularDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11593: Add support for covariance matrices to the cov Stream Evaluator (Joel
Bernstein)

* SOLR-10680: Add minMaxScale Stream Evaluator (Joel Bernstein)

* SOLR-11599: Change normalize function to standardize and make it work with

matrices (Joel Bernstein)

* SOLR-11602: Add Markov Chain Stream Evaluator (Joel Bernstein)

* SOLR-11607: Add grandSum, sumRows, sumColumns, scalarDivide, scalarMultiply,

scalarAdd,
scalarSubtract Stream Evaluators (Joel Bernstein)

* SOLR-11571: Add diff Stream Evaluator to support time series differencing (Mathew
Skaria, Joel Bernstein)

* SOLR-11570: Add support for correlation matrices to the corr Stream Evaluator
(Joel Bernstein)

* SOLR-11569: Add support for distance matrices to the distance Stream Evaluator
(Joel Bernstein)

* SOLR-11674: Support ranges in the probability Stream Evaluator (Joel Bernstein)

* SOLR-11680: Add normalizeSum Stream Evaluator (Joel Bernstein)

* SOLR-11697: Add geometricDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11485: Add olsRegress, spline and derivative Stream Evaluators (Joel

Bernstein)

Bug Fixes
----------------------

* SOLR-11423: Overseer queue needs a hard cap (maximum size) that clients respect
(Scott Blum, Joshua Humphries, Noble Paul)

* SOLR-11445: Overseer should not hang when process bad message. (Cao Manh Dat,
shalin)

* SOLR-11447: ZkStateWriter should process commands in atomic. (Cao Manh Dat,

shalin)

* SOLR-11491: Support retrieval of cluster properties in HttpClusterStateProvider.

(ab)

* SOLR-11146: Small fixes to the mapping functions in the Analytics Component

(Houston Putman)

* SOLR-11326: A bootstrap of a target cluster does not need to download the tlog
files from the source cluster
(Amrit Sarkar, Varun Thacker)

* SOLR-10874: Fix AIOOBE caused by FloatPayloadValueSource calling

PostingsEnum.nextPosition()
more than the given term's frequency in overridden FloatDocValues.floatVal().
(Michael Kosten, Erik Hatcher, Steve Rowe)

* SOLR-11484: CloudSolrClient does not invalidate cache or retry for RouteException

(noble, hossman)

* SOLR-11532: Solr hits NPE when fl only contains DV fields and any of them is a
spatial field. (Cao Manh Dat, David Smiley)

* SOLR-11557: Fix SolrZkClient.checkInterrupted (Tomás Fernández Löbbe)

* SOLR-11413: SolrGraphiteReporter fails to report metrics due to non-thread safe

code. (Erik Persson, ab)

* SOLR-9440: The ZkStateReader.removeCollectionStateWatcher method can cache a

DocCollection reference and
never update it causing stale state to be returned in ClusterState. (shalin)

* SOLR-11560: Specifying the replicationFactor parameter while restoring a

collection would lead to extra tlog
and pull replicas being created (Peter Szantai-Kis, Varun Thacker)

* SOLR-11586: Restored collection should use stateFormat=2 instead of 1. (Varun

Thacker)

* SOLR-11503: Collections created with legacyCloud=true cannot be opened if

legacyCloud=false (Erick Erickson)

* SOLR-11619: V2 requests that needed to be forwarded to other nodes would get an

NPE. (David Smiley)

* SOLR-11231: Guard against unset fields when performing language detection.

(Chris Beer via Steve Rowe)

* SOLR-11553: JSON Facet API: facet refinement could use UIF in phase one and DV
uninversion in phase 2, resulting
in too much memory use. (yonik)

* SOLR-11645: When the java commandline had exact duplicate arguments, the
admin UI dashboard would not display any commandline arguments.
(Webster Homer via Shawn Heisey)

* SOLR-11608: Correctly parse the new core-name in the V2 core rename API.
(Jason Gerlowski via Anshum Gupta)
* SOLR-11256: The queue size for ConcurrentUpdateSolrClient should default to 10
instead of throwing an
IllegalArgumentException. (Jason Gerlowski, Anshum Gupta)

* SOLR-11616: Snapshot the segments more robustly such that segments created during
a backup does does not fail the
operation (Varun Thacker)

* SOLR-11687: SolrCore.getNewIndexDir falsely returns {dataDir}/index on any

IOException reading index.properties
(Nikolay Martynov, Erick Erickson)

* SOLR-9137: bin/solr script ignored custom STOP_PORT on shutdown.

(Joachim Kohlhammer, Steve Rowe, Christine Poerschke)

* SOLR-11458: Improve error handling in MoveReplicaCmd to avoid potential loss of

data. (ab)

* SOLR-11590: Synchronize ZK connect/disconnect handling so that they are processed

in linear order
(noble, Varun Thacker)

* SOLR-11664: JSON Facet API: range facets containing unique, hll, min, max
aggregations over string fields
produced incorrect results since 7.0 (Volodymyr Rudniev, yonik)

* SOLR-11691: V2 requests for create-alias didn't work when the collections param
was an array.
(Jason Gerlowski, Gus Heck, David Smiley, noble)

Optimizations
----------------------
* SOLR-11285: Refactor autoscaling framework to avoid direct references to
Zookeeper and Solr
to make testing and large scale simulations easier. (ab)

* SOLR-11443: Remove the usage of workqueue for Overseer. (Cao Manh Dat, Scott
Blum)

* SOLR-11320: Lock autoscaling triggers when changes they requested are being made.
This helps to
ensure that cluster is in a stable state before processing any new trigger
events. (ab)

* SOLR-11641: Change `frange` to default to `cost=100` so default behavior is to

PostFilter if user specifies
`cache=false` (hossman)

* SOLR-11595: SolrIndexSearcher.localCollectionStatistics is now faster by

leveraging an existing
cached MultiFields. Noticeable when many fields are searched. (David Smiley)

Other Changes
----------------------
* SOLR-11478: Solr should remove itself from live_nodes in zk immediately on
shutdown. (Cao Manh Dat)

* SOLR-11145: Adds comprehensive unit tests for the Analytics Component v2 (Houston
Putman)
* SOLR-11389: For Solr(Shard|Cluster)Reporter instances the
SolrMetricManager.registerReporter
method is now called after the SolrCore or CoreContainer has been set for the
instance.
(Christine Poerschke)

* SOLR-11464, SOLR-11493, SOLR-11511: Minor refactorings to

DistributedUpdateProcessor. (Gus Heck, David Smiley)

* SOLR-11444: Improved consistency of collection alias handling, and collection

references that are
comma delimited lists of collections, across the various places collections can
be referred to.
Updates are delivered to the first in a list. It's now possible to refer to a
comma delimited list
of them in the path, ex: /solr/colA,colB/select?... (David Smiley)

* SOLR-11165: Ref guide documentation for Autoscaling APIs, triggers, listeners,

actions for Solr 7.1
(ab, Cassandra Targett, Noble Paul, shalin)

* SOLR-11562: Restore Solr logo ASCII-art in startup log by removing unnecessary

default confdir logging (janhoy)

* SOLR-11380: SolrJ must stream docs to server instead of writing to a buffer first
(noble)

* SOLR-11603: Remove unused (public) LTRScoringModel.hasParams() method. (Christine

Poerschke)

* SOLR-11606: Disable tests automatically if Mockito does not work with Java
runtime (Java 10).
(Uwe Schindler)

* SOLR-11618: Tone down verbosity of BackupManager logging (Varun Thacker)

* SOLR-11621: Fix spurious failures of TriggerIntegrationTest due to timing issues.

(shalin)

* SOLR-11628: Add documentation of maxRamMB for filter cache and query result
cache. (shalin)

* SOLR-11610: Refactored payload handling to use lucene's PayloadDecoder

framework (Alan Woodward)

* SOLR-9120: Reduce log level for inconsequential NoSuchFileException that

LukeRequestHandler may encounter (hossman)

* SOLR-10469: Move CloudSolrClient.setParallelUpdates to its Builder. (David

Smiley)

* SOLR-11507: SOLR-11638: Randomize SolrTestCaseJ4.CloudSolrClientBuilder more, and

simplify it.
(Jason Gerlowski, David Smiley)

* SOLR-11291: Factor out abstract metrics/SolrCore[Container]Reporter classes.

(Omar Abdelnabi, Christine Poerschke)
* SOLR-11713: Fixed CdcrUpdateLogTest.testSubReader() failure which was a test bug
(Varun Thacker, Amrit Sarkar)

================== 7.1.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.16
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.20.v20170531

Detailed Change List

----------------------

Upgrade Notes
----------------------

* 'autoAddReplicas' feature is ported to autoscaling framework. Existing users of

this feature should not have
to change anything. Note these changes:
1. Behaviour: Changing the autoAddReplicas property from disabled to enabled
using MODIFYCOLLECTION API
no longer replaces down replicas for the collection immediately. Instead,
replicas are only added
if a node containing them went down while autoAddReplicas was enabled. The params
autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay are no
longer used.
2. Deprecations: Enabling/disabling autoAddReplicas cluster wide with the API
will be deprecated; use
suspend/resume trigger APIs with name='.auto_add_replicas' instead.

* SOLR-11195: shard and cluster metric reporter configuration now requires a class
attribute.
If a reporter configures the group="shard" attribute then please also configure
the
class="org.apache.solr.metrics.reporters.solr.SolrShardReporter" attribute.
If a reporter configures the group="cluster" attribute then please also configure
the
class="org.apache.solr.metrics.reporters.solr.SolrClusterReporter" attribute.

* SOLR-11254: the abstract DocTransformer class now has an abstract score-less

transform method variant.

* SOLR-11283: all Stream Evaluators in solrj.io.eval have been refactored to have a

simplier and more
robust structure. This simplifies and condenses the code required to implement a
new Evaluator and
makes it much easier for evaluators to handle differing data types (primitives,
objects, arrays,
lists, and so forth). (Dennis Gove)

* SOLR-10962: in the ReplicationHandler the master.commitReserveDuration sub-

element is deprecated. Instead
 please configure a direct commitReserveDuration element for use in all modes
(master, slave, cloud).

* SOLR-11482: RunExecutableListener was removed for security reasons. If you want

to listen to
events caused by updates, commits, or optimize, write your own listener as native
Java class
as part of a Solr plugin.

* SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... })

the resolving of external entities is now disallowed by default.

New Features
----------------------
* SOLR-10339: New set-trigger and remove-trigger APIs for autoscaling. (shalin)

* SOLR-10340: New set-listener and remove-listener API for autoscaling. (shalin)

* SOLR-10358: New suspend-trigger and resume-trigger APIs for autoscaling. (shalin)

* SOLR-10376: Implement autoscaling trigger for nodeAdded event. (shalin)

* SOLR-10396: Implement trigger support for nodeLost event type (Cao Manh Dat,
shalin)

* SOLR-10496: New ComputePlanAction for autoscaling which uses the policy framework
to compute cluster
operations upon a trigger fire. (Noble Paul, shalin)

* SOLR-10965: New ExecutePlanAction for autoscaling which executes the operations

computed by ComputePlanAction
against the cluster. (shalin)

* SOLR-11019: Add addAll Stream Evaluator (Joel Bernstein)

* SOLR-10996: Implement TriggerListener API (ab, shalin)

* SOLR-11046: Add residuals Stream Evaluator (Joel Bernstein)

* SOLR-10858: Make UUIDUpdateProcessorFactory as Runtime URP (Amit Sarkar, noble)

* SOLR-11031: Implement SystemLogListener for autoscaling (ab)

* SOLR-11205: Any metrics value can be directly accessed in autoscaling policies

(noble)

* SOLR-11199: Payloads supports an "operator" param. Supported operators are 'or',

"phrase" ( default ).
A new "sum" function is also added. Example :
{!payload_score f=payload_field func=sum operator=or}A B C" (Varun Thacker)

* SOLR-11215: Make a metric accessible through a single param. (ab)

* SOLR-11244: Query DSL for Solr (Cao Manh Dat)

* SOLR-11317: JSON Facet API: min/max aggregations on numeric fields are now typed
better so int/long
fields return an appropriate integral type rather than a double. (yonik)
* SOLR-11316: JSON Facet API: min/max aggregations are now supported on single-
valued date fields.
(yonik)

* SOLR-10962: Make ReplicationHandler's commitReserveDuration configurable in

SolrCloud mode.
(Ramsey Haddad, Christine Poerschke, hossman)

* SOLR-11382: Lucene's Geo3D (surface of sphere & ellipsoid) is now supported on

spatial RPT fields by
setting spatialContextFactory="Geo3D". Furthermore, this is the first time Solr
has out of the box
support for polygons. (David Smiley)

* SOLR-11160: Add normalDistribution, uniformDistribution, sample and

kolmogorovSmirnov Stream Evaluators
(Joel Bernstein)

* SOLR-11225: Add cumulativeProbability Stream Evaluator (Joel Bernstein)

* SOLR-11241: Add discrete counting and probability Stream Evaluators (Joel

Bernstein)

* SOLR-11321: Add ebeAdd, ebeSubtract, ebeDivide, ebeMultiply, dotProduct and

cosineSimilarity Stream Evaluators
(Joel Bernstein)

* SOLR-11338: Add Kendall's Tau-b rank and Spearmans rank correlation Stream
Evaluators (Joel Bernstein)

* SOLR-11339: Add Canberra, Chebyshev, Earth Movers and Manhattan Distance Stream
Evaluators (Joel Bernstein)

* SOLR-11342: Add sumDifference and meanDifference Stream Evaluators (Joel

Bernstein)

* SOLR-11350: Add primes Stream Evaluator (Joel Bernstein)

* SOLR-11354: Add factorial and movingMedian Stream Evaluators (Joel Bernstein)

* SOLR-11377: Add expMovingAverage (exponential moving average) and

binomialCoefficient Stream Evaluators
(Mathew Skaria, Joel Bernstein)

* SOLR-11388: Add monteCarlo Stream Evaluator to support Monte Carlo simulations

(Joel Bernstein)

* SOLR-11398: Add weibullDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11400: Add logNormalDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11401: Add zipFDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11414: Add gammaDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11415: Add betaDistribution Stream Evaluator (Joel Bernstein)

* SOLR-11436: Add polyfit and polyfitDerivative Stream Evaluators (Joel Bernstein)

* SOLR-11439: Add harmonicFit Stream Evaluator (Joel Bernstein)

* SOLR-11076: New /autoscaling/history API to return past cluster events and

actions. (ab)

Bug Fixes
----------------------

* SOLR-10602: Triggers should be able to restore state from old instances when
taking over. (shalin)

* SOLR-10714: OverseerTriggerThread does not start triggers on overseer start until

autoscaling
config watcher is fired. (shalin)

* SOLR-10738: TriggerAction is initialised even if the trigger is never scheduled.

(shalin)

* SOLR-10668: fix NPE at sort=childfield(..) .. on absent values (Mikhail Khludnev)

* SOLR-8984: EnumField's error reporting to now indicate the field name in failure
log (Lanny Ripple,
Ann Addicks via Ishan Chattopadhyaya)

* SOLR-11012: Fix three (JavaBinCodec not being closed) Resource Leak warnings.
(Christine Poerschke)

* SOLR-11026: MoveReplicaSuggester must check if the target becomes more 'loaded'

than the source
if an operation is performed (noble)

* SOLR-10994: CREATE, CREATESHARD & ADDREPLICA to use policy framework & support
replica types (noble)

* SOLR-11011: Assign.buildCoreName can lead to error in creating a new core when

legacyCloud=false (Cao Manh Dat)

* SOLR-10944: Get expression fails to return EOF tuple (Susheel Kumar, Joel
Bernstein)

* SOLR-6086: Replica is active during autowarming resulting in queries being sent

to a replica that
may not have a registered searcher. This causes spikes in response times when
adding a replica
in busy clusters. (Ludovic Boutros, Timothy Potter, shalin)

* SOLR-11190: GraphQuery also supports string fields which are indexed=false and
docValues=true. Please refer to the
Javadocs for DocValuesTermsQuery for it's performance characteristics. (Karthik
Ramachandran, Varun Thacker)

* SOLR-11084: Issue with starting script with solr.home (-s) == solr (Leil Ireson,
Amrit Sarkar via Erick Erickson)

* SOLR-11255: Fix occasional ConcurrentModificationException when using

SolrInfoMBeanHandler. (ab)

* SOLR-11272: fix NPE when EmbeddedSolrServer handles /admin/* request and so one
(Stephen Allen via Mikhail Khludnev)
* SOLR-11289: fix comma handling in terms component. (yonik)

* SOLR-11164, SOLR-11180, SOLR-11220: Fix NullPointerException and always-returns-

zero
contrib/ltr OriginalScoreFeature issues in SolrCloud mode.
(Yuki Yano, Jonathan Gonzalez, Ryan Yacyshyn, Christine Poerschke)

* SOLR-11278: Stopping CDCR should cancel a running bootstrap operation. (Amrit

Sarkar, shalin)

* SOLR-11293: Potential data loss in TLOG replicas when masterVersion equals zero
(noble, Cao Manh Dat)

* SOLR-10101: TestLazyCores hangs (Erick Erickson)

* SOLR-11332: Fix sorting on 'enum' fieldTypes that use sortMissingFirst or

sortMissingLast (hossman)

* SOLR-11348: Fix the DIH database example (James Dyer)

* SOLR-11363: JSON Facet API: repeated values in a numeric points field with
docValues enabled
were double counted. (Hossman, yonik)

* SOLR-11085: Improve resiliency of autoscaling actions against overseer restarts

and operation failures. (shalin)

* SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is
trying to start some cores twice.
(Luiz Armesto, Shawn Heisey, Erick Erickson)

* SOLR-11399: The UnifiedHighlighter was ignoring the hl.fragsize parameter when

hl.bs.type=SEPARATOR
(Marc Morissette via David Smiley)

* SOLR-11224: SolrStream.close can hit an NPE (Erick Erickson)

* SOLR-11278: Fix a race condition in the CDCR bootstrap process which could lead
to bootstraps cancelling itself
(Amrit Sarkar, shalin, Varun Thacker)

* SOLR-11425: SolrClientBuilder does not allow infinite timeout (value 0). (Peter
Szantai-Kis via Mark Miller)

* SOLR-11449: MoveReplicaCmd mistakenly called registerCollectionStateWatcher on

failure. (ab)

* SOLR-11477: Disallow resolving of external entities in the XML query parser

(defType=xmlparser).
(Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke)

* SOLR-12444: Updating a cluster policy fails (noble)

Optimizations
----------------------

* SOLR-10985: Remove unnecessary toString() calls in solr-core's search package's

debug logging.
 (Michael Braun via Christine Poerschke)

* SOLR-11000: Changes made via AutoScalingHandler should be atomic. (ab, shalin)

* SOLR-11124: MoveReplicaCmd should skip deleting old replica in case of its node
is not live (Cao Manh Dat)

* SOLR-10769: Allow nodeAdded / nodeLost events to report multiple nodes in one

event. (ab)

* SOLR-11242: QueryParser: re-use the LookaheadSuccess exception. (Michael Braun

via David Smiley)

* SOLR-11314: FastCharStream: re-use the READ_PAST_EOF exception. (Michael Braun

via David Smiley)

* SOLR-8344: Decide default when requested fields are both column and row stored.
(Cao Manh Dat, David Smiley)

* SOLR-10285: Skip LEADER messages when there are leader only shards (Cao Manh Dat,
Joshua Humphries)

* SOLR-11424: When legacyCloud=false, cores should not publish itself as DOWN on

startup. (Cao Manh Dat)

Other Changes
----------------------
* SOLR-10643: Throttling strategy for triggers and policy executions. (shalin)

* SOLR-10764: AutoScalingHandler should validate policy and preferences before

updating zookeeper. (shalin)

* SOLR-10827: Factor out abstract FilteringSolrMetricReporter class. (Christine

Poerschke)

* SOLR-10957: Changed SolrCoreParser.init to use the resource loader from

getSchema()
instead of the resource loader from getCore(). (Christine Poerschke)

* SOLR-10748: Make stream.body configurable and disabled by default (janhoy)

* SOLR-10964: Reduce SolrIndexSearcher casting in LTRRescorer. (Christine

Poerschke)

* SOLR-11075: Refactor handling of params in CloudSolrStream and FacetStream (Erick

Erickson)

* SOLR-11052: Remove unnecessary Long-to-Integer and back casts in

ReplicationHandler.
(Ramsey Haddad via Christine Poerschke)

* SOLR-11106: TestLBHttpSolrClient.testReliablity takes 30 seconds because of the

wrong server name
(Kensho Hirasawa via Erick Erickson)

* SOLR-10338: Configure SecureRandom non blocking for tests. (Mihaly Toth, hossman,
Ishan Chattopadhyaya, via Mark Miller)

* SOLR-10916: Convert tests that extend LuceneTestCase and use MiniSolrCloudCluster

 to instead extend SolrCloudTestCase. (Steve Rowe)

* SOLR-11131: Document 'assert' as a command option in bin/solr, and bin/solr.cmd

scripts.
(Jason Gerlowski via Anshum Gupta)

* SOLR-11140: Remove unused parameter in (private)

SolrMetricManager.prepareCloudPlugins method.
(Omar Abdelnabi via Christine Poerschke)

* SOLR-11187: contrib/ltr TestModelManagerPersistence improvements. (Yuki Yano via

Christine Poerschke)

* SOLR-10397: Port 'autoAddReplicas' feature to the autoscaling framework and make

it work with non-shared filesystems
(Cao Manh Dat, shalin)

* SOLR-11157: remove-policy must fail if a policy to be deleted is used by a

collection (noble)

* SOLR-11090: Add Replica.getProperty accessor. (Christine Poerschke)

* SOLR-11061: Add a spins metric for data directory paths. (ab)

* SOLR-11071: Improve TestIntervalFacets.testRandom (Tomás Fernández Löbbe)

* SOLR-11195: Require class attribute for shard and cluster metric reporter
configuration. (Christine Poerschke)

* SOLR:10822: Share a Policy.session object between multiple collection admin calls

to get the correct computations (noble)

* SOLR-11249: Upgrade Jetty from 9.3.14.v20161028 to 9.3.20.v20170531 (Michael

Braun via David Smiley)

* SOLR-11270: Automatically added SystemLogListener should not prevent deleting a

trigger. (ab)

* SOLR-5129: Timeout property for waiting ZK get started. (Cao Manh Dat, Hrishikesh
Gadre, Varun Thacker)

* SOLR-10628: Less verbose output from bin/solr commands. (Jason Gerlowski, janhoy)

* SOLR-11240: Raise UnInvertedField internal limit. (Toke Eskildsen)

* SOLR-11254: Add score-less (abstract) DocTransformer.transform method. (Christine

Poerschke)

* SOLR-11209: Upgrade HttpClient to 4.5.3. (Hrishikesh Gadre via Mark Miller)

* SOLR-11322: JSON Facet API: instead of returning NaN, min & max aggregations omit
the value for any bucket with no values in the numeric field. (yonik)

* SOLR-10990: Breakup QueryComponent.process method for readability. (Christine

Poerschke)

* SOLR-11132: Refactor common getSortField logic in various FieldTypes (Jason

Gerlowski, hossman)
* SOLR-11351: Make LTRScoringModel model more extensible. (Christine Poerschke)

* SOLR-10451: Remove contrib/ltr/lib from lib includes in the techproducts example

config (sungjunyoung via janhoy)

* SOLR-11306: Fix inaccurate comments on docValues and StrField in the example

schemas
(Tom Burton-West, Jason Gerlowski, Varun Thacker)

* SOLR-11482: RunExecutableListener was removed for security reasons. (Michael

Stepankin,
Olga Barinova, Uwe Schindler, Tomás Fernández Löbbe)

* SOLR-10335: Upgrade to Tika 1.16. (Tim Allison, Tomás Fernández Löbbe via shalin)

================== 7.0.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.14.v20161028

Bug Fixes
----------------------

* SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is
trying to start some cores twice.
(Luiz Armesto, Shawn Heisey, Erick Erickson)

* SOLR-11406: Solr 7.0 cannot read indexes from 6.x versions. (Shawn Heisey, Steve
Rowe)

================== 7.0.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.14.v20161028

Upgrading from Solr 6.x

----------------------

* All Trie* numeric and date field types have been deprecated in favor of *Point
field types.
Point field types are better at range queries (speed, memory, disk), however
simple field:value queries underperform
relative to Trie. Either accept this, or continue to use Trie fields.
This shortcoming may be addressed in a future release.
* The default response type is now JSON ("wt=json") instead of XML, and line
indentation is now on by default
("indent=on"). If you expect the responses to your queries to be returned in the
previous format (XML
format, no indentation), you must now you must now explicitly pass in "wt=xml"
and "indent=off" as query
parameters, or configure them as defaults on your request handlers. See SOLR-
10494 for more details.

* the cluster property 'legacyCloud' is set to false from 7.0. This means
'zookeeper is the truth' by
default. If an entry for a replica does not exist in the state.json, that replica
cannot get
registered. This may affect users who use that feature where they bring up
replicas and they are
automatically registered as a part of a shard. However, it is possible to fall
back to the old behavior by
setting the property legacyCloud=true , in the cluster properties using the
following command

./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd clusterprop

-name legacyCloud -val true

* HttpClientInterceptorPlugin is now HttpClientBuilderPlugin and must work with a

SolrHttpClientBuilder rather than an HttpClientConfigurer.

* HttpClientUtil now allows configuring HttpClient instances via

SolrHttpClientBuilder
rather than an HttpClientConfigurer. Use of env variable
SOLR_AUTHENTICATION_CLIENT_CONFIGURER
no longer works, please use SOLR_AUTHENTICATION_CLIENT_BUILDER

* SolrClient implementations now use their own internal configuration for socket
timeouts,
connect timeouts, and allowing redirects rather than what is set as the default
when
building the HttpClient instance. Use the appropriate setters on the SolrClient
instance.

* HttpSolrClient#setAllowCompression has been removed and compression must be

enabled as
a constructor param.

* HttpSolrClient#setDefaultMaxConnectionsPerHost and
HttpSolrClient#setMaxTotalConnections have been removed. These now default very
high and can only be changed via param when creating an HttpClient instance.

* Query time join with scoring {!join score=none} (even none) doesn't handle single
value numeric fields.
Users are advised to convert these fields into string and reindex.

* Deprecated collection and configset methods on MiniSolrCloudCluster have been

removed

* Index-time boosts are not supported anymore. If any boosts are provided, they
will be ignored by the indexing chain. As a replacement, index-time scoring
factors should be indexed in a separate field and combined with the query
score using a function query.
* Deprecated method getNumericType() has been removed from FieldType. Use
getNumberType() instead

* MBean names and attributes now follow hierarchical names used in metrics. This is
reflected also in
/admin/mbeans and /admin/plugins output, and can be observed in the UI Plugins
tab, because now all these
APIs get their data from the metrics API. The old (mostly flat) JMX view has been
removed.

* <jmx> element in solrconfig.xml is no longer supported. Equivalent functionality

can be configured in
solr.xml using <metrics><reporter ...> element and SolrJmxReporter
implementation. Limited back-compatibility
is offered by automatically adding a default instance of SolrJmxReporter if it's
missing, AND when a local
MBean server is found (which can be activated either via ENABLE_REMOTE_JMX_OPTS
in solr.in.sh or via system
properties, eg. -Dcom.sun.management.jmxremote). This default instance exports
all Solr metrics from all
registries as hierarchical MBeans. This behavior can be also disabled by
specifying a SolrJmxReporter
configuration with a boolean init arg "enabled" set to "false". For a more fine-
grained control users
should explicitly specify at least one SolrJmxReporter configuration.

* The sow (split-on-whitespace) request param now defaults to false (true in

previous versions).
This affects the edismax and standard/"lucene" query parsers: if the sow param is
not specified,
query text will not be split on whitespace before analysis. See
https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-
support/ .

* Setting <solrQueryParser defaultOperator="..."/> in schema is no longer allowed

and will cause an exception.
Please use "q.op" parameter on the request instead. For more details, see SOLR-
10584.

* Setting <defaultSearchField> in schema is no longer allowed and will cause an

exception.
Please use "df" parameter on the request instead. For more details, see SOLR-
10585.

* The PostingsSolrHighlighter is deprecated. Furthermore, it now internally works

via a re-configuration
of the UnifiedSolrHighlighter.

* Deprecated LatLonType, GeoHashField, SpatialPointVectorFieldType, and

SpatialTermQueryPrefixTreeFieldType.
Instead, switch to LatLonPointSpatialField or SpatialRecursivePrefixTreeFieldType
or RptWithGeometrySpatialField.

* The 'lucenePlusSort' QParser has been deprecated, and is no longer implicitly

defined. If you wish to continue using
this QParser untill Solr 8, you must explicitly register it in your
solrconfig.xml:
'<queryParser name="lucenePlusSort" class="solr.OldLuceneQParserPlugin"/>'
* In solrconfig.xml the deprecated <mergePolicy> and <mergeFactor> and
<maxMergeDocs> elements have
been removed in favor of the <mergePolicyFactory> element introduced by SOLR-8621
in Solr 5.5.0.

* Custom SolrMetricReporter classes must implement a new, factored out doInit()

method and the validate() method
must pass for all reporters including reporters that are not enabled, please see
SOLR-10671 for details.

* TemplateUpdateRequestProcessorFactory uses {} instead of ${} for template

* SOLR-9565: The name of TemplateUpdateRequestProcessorFactory' is changed to

'template' from 'Template' and the
name of 'AtomicUpdateProcessorFactory' is changed to 'atomic' from 'Atomic'

* The default for eDismax parameter 'lowercaseOperators' now depends on the

luceneMatchVersion setting in solrconfig.
It remains 'true' for luceneMatchVersion < 7.0.0 and changes to 'false' from
7.0.0. See also SOLR-4646

* The unused 'valType' option has been removed from ExternalFileField, if you have
this in your schema you
can safely remove it. see SOLR-10929 for more details.

* Config sets basic_configs and data_driven_schema_configs have now been merged

into _default. It has data driven nature
enabled by default, and can be turned off (after creating a collection) with:
curl http://host:8983/solr/mycollection/config -d '{"set-user-property":
{"update.autoCreateFields":"false"}}'
Please see SOLR-10574 for details.

* The data driven config (now _default) for auto-creating fields earlier defaulted
to "string" for text input.
Now the default is to use "text_general" and to add a copyField to the schema,
copying to a "*_str" dynamic field,
with a cutoff at 256 characters. This enables full text search as well as
faceting. See SOLR-9526 for more.

* SOLR-10123: The Analytics Component has been upgraded to support distributed

collections, expressions over multivalued
fields, a new JSON request language, and more. DocValues are now required for any
field used in the analytics expression
whereas previously docValues was not required. Please see SOLR-10123 for details.

* solrconfig.xml: <requestDispatcher handleSelect="..."> now defaults to false when

luceneMatchVersion >= 7.0,
thus ignoring "qt". Regardless, "qt" is still used as a SolrJ special parameter
that specifies the request handler
(tail URL path) to use. If you have request handlers without a leading '/', you
can set handleSelect="true"
or better yet consider migrating.

* StandardRequestHandler is deprecated. Simply use SearchHandler instead.

* The parameter names 'fromNode' for MOVEREPLICA and 'source', 'target' for
REPLACENODE have been deprecated and
replaced with 'sourceNode' and 'targetNode' instead. The old names will continue
to work for back-compatibility
but they will be removed in 8.0. See SOLR-11068 for more details.

* All deperated methods of ClusterState (except getZkClusterStateVersion())

have been removed. Use DocCollection methods instead.

* SOLR-11023: EnumField has been deprecated in favor of new EnumFieldType.

* SOLR-11239: The use of maxShardsPerNode is not supported when a cluster policy is

in effect or
when a collection specific policy is specified during collection creation.

* V2 APIs are now available at /api, in addition to /v2 (which is now deprecated).
Legacy APIs continue to remain
available at /solr.

* Solr was tested and is compatible with the final release candidate of Java 9. All
startup scripts
detect Java 9 correctly and setup Garbage Collector logging. If the configuration
file contains
logging options that are no longer supported with Java 9, startup will fail.

* SOLR-10307: If starting Jetty without the Solr start script, you must now pass
keystore and truststore
passwords via the env variables SOLR_SSL_KEY_STORE_PASSWORD and
SOLR_SSL_TRUST_STORE_PASSWORD rather
than system properties.

* SOLR-10379: ManagedSynonymFilterFactory has been deprecated in favor of

ManagedSynonymGraphFilterFactory.

* SOLR-10503: CurrencyField has been deprecated in favor of new CurrencyFieldType.

New Features
----------------------
* SOLR-9857, SOLR-9858: Collect aggregated metrics from nodes and shard leaders in
overseer. (ab)

* SOLR-9835: Create another replication mode for SolrCloud

* SOLR-10356: Adds basic math Streaming Evaluators (Dennis Gove)

* SOLR-10393: Adds UUID Streaming Evaluator (Dennis Gove)

* SOLR-10046: Add UninvertDocValuesMergePolicyFactory class. (Keith Laban,

Christine Poerschke)

* SOLR-10547: JSON Facet API: Implement support for single-valued string fields for
min/max aggregations.
(yonik)

* SOLR-10262: Add support for configurable metrics implementations. (ab)

* SOLR-10431: Make it possible to invoke v2 api calls using SolrJ (Cao Manh Dat,
Noble Paul, shalin)

* SOLR-10233: Add support for different replica types, that can handle updates
differently:
- NRT: Writes updates to transaction log and indexes locally. Replicas of type
“NRT” support NRT
(soft commits) and RTG. Any NRT replica can become a leader. This was the
only type supported
in SolrCloud until now and it’s the default type.
- TLOG: Writes to transaction log, but not to index, uses replication to copy
segment files from the
shard leader. Any TLOG replica can become leader (by first applying all
local transaction log
elements). If a replica is of type TLOG but is also the leader, it will
behave as a NRT. This
is exactly what was added in SOLR-9835 (non-realtime replicas), just the
API and naming changes.
- PULL: Doesn’t index or writes to transaction log, just replicates from the
shard leader. PULL replicas
can’t become shard leaders (i.e., if there are only PULL replicas in the
collection at some point,
updates will fail same as if there is no leaders, queries continue to
work), so they don’t even
participate in elections.
(Tomás Fernández Löbbe)

* SOLR-10278: A new DSL to set cluster-wide preferences and policies on how to

allocate replicas to nodes
(noble, shalin)

* SOLR-10373: Implement read API for autoscaling configuration at

/admin/autoscaling or
/cluster/autoscaling paths. (shalin)

* SOLR-10677: Expose a diagnostics API to return nodes sorted by load in descending

order and
any policy violations. (shalin)

* SOLR-7452: Refinement for JSON Facet API: Adding refine:true to any terms facet
will cause an
additional distributed search phase (overlapped with field retrieval) that
requests additional info
for top facet buckets from shards that did not previously contribute to that
bucket.
This will correct counts (and other statistics) for those top buckets collected
in the first
phase. (yonik)

* SOLR-10433: CollectionAdmin requests in SolrJ to support V2 calls (noble)

* SOLR-9989: Add support for PointFields in FacetModule (JSON Facets) (Cao Manh
Dat)

* SOLR-10406: v2 API error messages list the URL request path as /solr/____v2/...
when the original path was /v2/... (Cao Manh Dat, noble)

* SOLR-10574: New _default config set replacing basic_configs and

data_driven_schema_configs.
(Ishan Chattopadhyaya, noble, shalin, hossman, David Smiley, janhoy, Alexandre
Rafalovitch)

* SOLR-10272: Use _default config set if no collection.configName is specified with

CREATE (Ishan Chattopadhyaya)
* SOLR-9526: Data driven schema now indexes text field "foo" as both "foo"
(text_general) and as "foo_str" (string)
to facilitate both search and faceting. AddSchemaFieldsUpdateProcessor now has
the ability to add a "copyField" to
the type mappings, with an optional maxChars limitation. You can also define one
typeMappings as default.
This also solves issues SOLR-8495, SOLR-6966, and SOLR-7058
(janhoy, Steve Rowe, hossman, Alexandre Rafalovitch, Shawn Heisey, Cao Manh Dat)

* SOLR-10123: Upgraded the Analytics Component to version 2.0 which now supports
distributed collections, expressions over
multivalued fields, a new JSON request language, and more. DocValues are now
required for any field used in the analytics
expression whereas previously docValues was not required. Please see SOLR-10123
for details. (Houston Putman)

* SOLR-10282: bin/solr support for enabling Kerberos authentication (Ishan

Chattopadhyaya, Jason Gerlowski)

* SOLR-11093: Add support for PointFields for {!graph} query. (yonik)

* SOLR-10845: Add support for PointFields to {!graphTerms} query that is internally

used by some graph traversal streaming expressions. (yonik)

* SOLR-10939: Add support for PointsFields to {!join} query. Joined fields should
also have docValues enabled. (yonik)

* SOLR-11173 TermsComponent support for Points fields. (yonik)

* SOLR-10849: MoreLikeThisComponent should expose setMaxDocFreqPct (maxDoc

frequency percentage). (Dawid Weiss)

* SOLR-10307: Allow Passing SSL passwords through environment variables. (Mano

Kovacs, Michael Suzuki via Mark Miller)

* SOLR-10379: Add ManagedSynonymGraphFilterFactory, deprecate

ManagedSynonymFilterFactory. (Steve Rowe)

* SOLR-10479: Adds support for

HttpShardHandlerFactory.loadBalancerRequests(MinimumAbsolute|MaximumFraction)
configuration. (Ramsey Haddad, Daniel Collins, Christine Poerschke)

* SOLR-3702: concat(...) function query (Andrey Kudryavtsev via Mikhail Khludnev)

* SOLR-10767: Add movingAvg Stream Evaluator (Joel Bernstein)

* SOLR-10813: Add arraySort Stream Evaluator (Joel Bernstein)

* SOLR-10696: Add cumulative probability function (Joel Bernstein)

* SOLR-10765: Add anova Stream Evaluator (Joel Bernstein)

* SOLR-10754: Add hist Stream Evaluator (Joel Bernstein)

* SOLR-10753: Add array Stream Evaluator (Joel Bernstein)

* SOLR-10747: Allow /stream handler to execute Stream Evaluators directly (Joel

Bernstein)
* SOLR-10743: Add sequence StreamEvaluator (Joel Bernstein)

* SOLR-10684: Add finddelay Stream Evaluator (Joel Bernstein)

* SOLR-10731: Add knn Streaming Expression (Joel Bernstein)

* SOLR-10724: Add describe Stream Evaluator (Joel Bernstein)

* SOLR-10693: Add copyOfRange Stream Evaluator (Joel Bernstein)

* SOLR-10623: Add sql Streaming Expression (Joel Bernstein)

* SOLR-10661: Add copyOf Stream Evaluator (Joel Bernstein)

* SOLR-10663: Add distance Stream Evaluator (Joel Bernstein)

* SOLR-10664: Add scale Stream Evaluator (Joel Bernstein)

* SOLR-10666: Add rank transformation Stream Evaluator (Joel Bernstein)

* SOLR-10662: Add length Stream Evaluator (Joel Bernstein)

* SOLR-10660: Add reverse Stream Evaluator (Joel Bernstein)

* SOLR-9910: Add solr/solr.cmd parameter to append jetty parameters to the start

script.
(Mano Kovacs via Mark Miller)

* SOLR-6671: Possible to set solr.data.home property as root dir for all data
(janhoy, Shawn Heisey, Mark Miller)

Bug Fixes
----------------------
* SOLR-9262: Connection and read timeouts are being ignored by UpdateShardHandler
after SOLR-4509.
(Mark Miller, shalin)

* SOLR-9837: Fix 55% performance regression of FieldCache uninvert time of

numeric fields. (yonik)

* SOLR-10408: v2 API introspect should return useful message for non-existent

command (Cao Manh Dat)

* SOLR-10411: v2 Collection API "modify" command specification's replicationFactor

property is incorrectly typed as string,
should be integer (Cao Manh Dat)

* SOLR-10405: v2 API introspect response contains multiple copies of the

experimental format WARNING (Cao Manh Dat)

* SOLR-10412: v2 API: many API command specification properties are typed "number"
but should instead be typed "integer"
(Steve Rowe, Cao Manh Dat)

* SOLR-10413: v2 API: parsed JSON type should be coerced to expected type (Cao Manh
Dat, Noble Paul)

* SOLR-10223: Allow running examples as root on Linux with -force option (janhoy)
* SOLR-10830: Solr now correctly enforces that the '_root_' field has the same
fieldType as the
uniqueKey field. With out this enforcement, child document updating was
unreliable. (hossman)

* SOLR-10876: Regression in loading runtime UpdateRequestProcessors like

TemplateUpdateProcessorFactory (noble)

* SOLR-10886: Using V2Request.process(solrClient) method throws NPE if the API

returns an error (Cao Manh Dat)

* SOLR-10921: Work around the static Lucene BooleanQuery.maxClauseCount that causes

Solr's maxBooleanClauses
setting behavior to be "last core wins". This patch sets
BooleanQuery.maxClauseCount to its maximum value,
thus disabling the global check, and replaces it with specific checks where
desired via
QueryUtils.build(). (yonik)

* SOLR-10948: Fix extraction component to treat DatePointField the same as

TrieDateField (hossman)

* SOLR-10506: Fix memory leak (upon collection reload or ZooKeeper session expiry)
in ZkIndexSchemaReader.
(Torsten Bøgh Köster, Christine Poerschke, Jörg Rathlev, Mike Drob)

* SOLR-6807: CloudSolrClient's ZK state version check with the server was ignored
when handleSelect=false
(David Smiley)

* SOLR-10878: MOVEREPLICA command may lose data when replicationFactor is 1. (ab,

shalin)

* SOLR-10879: DELETEREPLICA and DELETENODE commands should prevent data loss when
replicationFactor is 1. (ab)

* SOLR-10983: Fix DOWNNODE -> queue-work explosion (Scott Blum, Joshua Humphries)

* SOLR-10826: Fix CloudSolrClient to expand the collection parameter correctly (Tim

Owen via Varun Thacker)

* SOLR-11039: Next button in Solr admin UI for collection list pagination does not
work. (janhoy)

* SOLR-11041: MoveReplicaCmd do not specify ulog dir in case of HDFS (Cao Manh Dat)

* SOLR-11045: The new replica created by MoveReplica will have to have same name
and coreName as the
old one in case of HDFS (Cao Manh Dat)

* SOLR-11043: Fix facet.range.method=dv and interval facets on single-valued float

fields with negative values.
(Tomás Fernández Löbbe, Steve Rowe)

* SOLR-11073: Fix overflow in interval faceting when querying Long limits (e.g.
(Long.MAX_VALUE TO Long.MAX_VALUE])
(Tomás Fernández Löbbe)

* SOLR-11057: Fix overflow in point range queries when querying the type limits
 (e.g. q=field_i:{Integer.MAX_VALUE TO Integer.MAX_VALUE]) (Tomás Fernández Löbbe)

* SOLR-11136: Fix solrj XMLResponseParser when nested docs transformer is used with
indented XML (hossman)

* SOLR-11130: V2Request in SolrJ should return the correct collection name so that
the request is forwarded to the
correct node (noble)

* SOLR-11151: SolrInfoMBeanHandler.getDiff() ADD case non-functional: NPE when a

bean value goes from null -> non-null.
(Steve Rowe)

* SOLR-11154: Child documents' return fields now include useDocValuesAsStored

fields (Mohammed Sheeri Shaketi Nauage via
Ishan Chattopadhyaya)

* SOLR-11163: Fix contrib/ltr Normalizer persistence after solr core reload or

restart.
(Yuki Yano via Christine Poerschke)

* SOLR-11182: A split shard failure on IOException should be logged (Varun Thacker)

* SOLR-10353: TestSQLHandler reproducible failure: No match found for function

signature min(<NUMERIC>) (Kevin Risden)

* SOLR-11221: SolrJmxReporter broken on core reload. This resulted in some or most

metrics not being reported
via JMX after core reloads, depending on timing. (ab)

* SOLR-11235: Some SolrCore metrics should check if core is closed before

reporting. (ab)

* SOLR-10698: StreamHandler should allow connections to be closed early (Joel

Bernstein, Varun Thacker, Erick Erickson)

* SOLR-11243: Replica Placement rules are ignored if a cluster policy exists.

(shalin)

* SOLR-11268: AtomicUpdateProcessor complains missing UpdateLog (noble, Ishan

Chattopadhyaya)

* SOLR-8689: Fix bin/solr.cmd so it can run properly on Java 9 (Uwe Schindler,

hossman)

* SOLR-10723 JSON Facet API: resize() implemented incorrectly for CountSlotAcc,

HllAgg.NumericAcc
resulting in exceptions when using a hashing faceting method and sorting by
hll(numeric_field).
(yonik)

* SOLR-10719: Creating a core.properties fails if the parent of core.properties is

a symlinked dierctory
(Erick Erickson)

* SOLR-10360: Solr HDFS snapshot export fails due to FileNotFoundException error

when using MR1 instead of
yarn. (Hrishikesh via Mark Miller)
* SOLR-10137: Ensure that ConfigSets created via API are mutable. (Hrishikesh via
Mark Miller)

* SOLR-10829: Fixed IndexSchema to enforce that uniqueKey can not be Points based
for correctness (hossman)

* SOLR-10836: The query parsers igain, significantTerms, and tlogit (used by

streaming expressions by
the same name) might throw a NullPointerException if the referenced field had no
indexed data in some
shards. The fix included an optimization to use Solr's cached AtomicReader
instead of re-calculating.
(David Smiley)

* SOLR-10715: /v2/ should not be an alias for /v2/collections (Cao Manh Dat)

* SOLR-10835: Add support for point fields in Export Handler (Tomás Fernández
Löbbe)

* SOLR-10704: REPLACENODE may cause data loss when replicationFactor is 1. (ab,

shalin)

* SOLR-10833: Point numeric fields should throw SolrException(BAD_REQUEST) for

malformed numbers in queries.
Trie numeric fields should throw SolrException(BAD_REQUEST) for
malformed docValues range queries.
(hossman, Tomás Fernández Löbbe)

* SOLR-10832: Fixed VersionInfo.getMaxVersionFromIndex when using PointsField with

indexed="true" (hossman)

* SOLR-10763: Admin UI replication tab sometimes empty when failed replications

(janhoy, Bojan Vitnik)

* SOLR-10824: fix NPE ExactSharedStatsCache, fixing maxdocs skew for terms which
are absent at one of shards
when using one of Exact*StatsCache (Mikhail Khludnev)

* SOLR-10963: Fix example json in MultipleAdditiveTreesModel javadocs.

(Stefan Langenmaier via Christine Poerschke)

* SOLR-10914: RecoveryStrategy's sendPrepRecoveryCmd can get stuck for 5 minutes if

leader is unloaded. (shalin)

* SOLR-11198: downconfig downloads empty file as folder (Erick Erickson)

Optimizations
----------------------

* SOLR-4509: Move to non deprecated HttpClient impl classes to remove stale

connection
check on every request and move connection lifecycle management towards the
client.
(Ryan Zezeski, Mark Miller, Shawn Heisey, Steve Davids)

* SOLR-9255: Rename SOLR_AUTHENTICATION_CLIENT_CONFIGURER ->

SOLR_AUTHENTICATION_CLIENT_BUILDER (janhoy)

* SOLR-9579: Make Solr's SchemaField implement Lucene's IndexableFieldType,

removing the
creation of a Lucene FieldType every time a field is indexed. (John Call, yonik)

* SOLR-9822: JSON Facet API: Recover performance lost due to the DocValues
transition to
an iterator API (LUCENE-7407). This only fixes calculating counts for single-
valued
string fields from the FieldCache, resulting in up to 56% better throughput for
those cases.
(yonik)

* SOLR-10727: Avoid polluting the filter cache for certain types of faceting
(typically ranges) when
the base docset is empty. (David Smiley)

* SOLR-11070: Make docValues range queries behave the same as Trie/Point fields for
Double/Float Infinity cases
(Tomás Fernández Löbbe, Andrey Kudryavtsev)

* SOLR-10634: JSON Facet API: When a field/terms facet will retrieve all buckets
(i.e. limit:-1)
and there are no nested facets, aggregations are computed in the first collection
phase
so that the second phase which would normally involve calculating the domain for
the bucket
can be skipped entirely, leading to large performance improvements. (yonik)

* SOLR-10722: Speed up Solr's use of the UnifiedHighlighter be re-using FieldInfos.

(David Smiley)

Other Changes
----------------------
* SOLR-10236: Removed FieldType.getNumericType(). Use getNumberType() instead.
(Tomás Fernández Löbbe)

* SOLR-10347: Removed index level boost support from "documents" section of the
admin UI (Amrit Sarkar via
Tomás Fernández Löbbe)

* SOLR-9959: SolrInfoMBean category and hierarchy cleanup. Per-component statistics

are now obtained from
the metrics API, legacy JMX support has been replaced with SolrJmxReporter
functionality. Several reporter
improvements (support for multiple prefix filters, "enabled" flag, reuse of
service clients). (ab)

* SOLR-10418: Expose safe system properties via metrics API as 'system.properties'

in 'solr.jvm' group.
Add support for selecting specific properties from any compound metric using
'property' parameter to
/admin/metrics handler. (ab)

* SOLR-10557: Make "compact" format default for /admin/metrics. (ab)

* SOLR-10310: By default, stop splitting on whitespace prior to analysis

in edismax and standard/"lucene" query parsers. (Steve Rowe)

* SOLR-10647: Move the V1 <-> V2 API mapping to SolrJ (noble)

* SOLR-10584: We'll now always throw an exception if defaultOperator is found in
schema. This config has
been deprecated since 3.6, and enforced for new configs since 6.6 (janhoy)

* SOLR-10585: We'll now always throw an exception if defaultSearchField is found in

schema. This config has
been deprecated since 3.6, and enforced for new configs since 6.6 (janhoy, David
Smiley)

* SOLR-10414: RecoveryStrategy is now a Runnable instead of a Thread. (Tomás

Fernández Löbbe)

* SOLR-10042: Delete old deprecated Admin UI, leaving the AngularJS UI the only one
supported (janhoy)

* SOLR-10378: Clicking Solr logo on AdminUI shows blank page (Takumi Yoshida via
janhoy)

* SOLR-10700: Deprecated and converted the PostingsSolrHighlighter to extend

UnifiedSolrHighlighter and thus no
longer use the PostingsHighlighter. It should behave mostly the same. (David
Smiley)

* SOLR-10710: Fix LTR failing tests. (Diego Ceccarelli via Tomás Fernández Löbbe)

* SOLR-10755: delete/refactor many solrj deprecations (hossman)

* SOLR-10752: replicationFactor (nrtReplicas) default is 0 if tlogReplicas is

specified when creating a collection
(Tomás Fernández Löbbe)

* SOLR-10757: delete/refactor/cleanup CollectionAdminRequest deprecations (hossman)

* SOLR-10793: BlobHandler should have a well-known permission name (noble)

* SOLR-10744: Update noggit to newer version (0.8) (noble)

* SOLR-10792: Deprecate and remove implicit registration of "lucenePlusSort" aka

OldLuceneQParser (hossman)

* SOLR-10791: Remove deprecated options in SSLTestConfig (hossman)

* LUCENE-7852: Correct copyright year(s) in solr/LICENSE.txt file.

(Christine Poerschke, Steve Rowe)

* SOLR-8668: In solrconfig.xml remove <mergePolicy> (and related <mergeFactor> and

<maxMergeDocs>)
support in favor of the <mergePolicyFactory> element introduced by SOLR-8621 in
Solr 5.5.0.
(Christine Poerschke, hossman)

* SOLR-10799: Extracted functionality to collect eligible replicas from

HttpShardHandler.prepDistributed()
to a new method (Domenico Fabio Marino via Tomás Fernández Löbbe)

* SOLR-10801: Remove several deprecated methods that were exposed to plugin writers
(hossman)

* SOLR-10671: Add abstract doInit method to the SolrMetricReporter base class.

 (Christine Poerschke, Anshum Gupta)

* SOLR-10713: Ignore .pid and .out files in solr working directory (Jason Gerlowski
via Mike Drob)

* SOLR-10764: AutoScalingHandler should validate policy and preferences before

updating zookeeper. (shalin)

* SOLR-10782: Improve error handling and tests for Snitch and subclasses and
general cleanups. (Noble Paul, shalin)

* SOLR-10419: All collection APIs should use the new Policy framework for replica
placement. (Noble Paul, shalin)

* SOLR-10800: Factor out HttpShardHandler.transformReplicasToShardUrls from

HttpShardHandler.prepDistributed.
(Domenico Fabio Marino, Christine Poerschke)

* SOLR-10779: JavaBinCodec should use close consistently rather than having

marshal() and close() call finish()
(which closes the underlying stream). (Erick Erickson)

* SOLR-9623: Disable remote streaming in example configs by default. Adjust Upload

Limit defaults (janhoy, yonik)

* SOLR-4646: eDismax lowercaseOperators now defaults to "false" for

luceneMatchVersion >= 7.0.0 (janhoy, David Smiley)

* SOLR-8256: Set legacyCloud=false as default (Cao Manh Dat)

* SOLR-10929: Removed unused 'valType' option from ExternalFileField (hossman)

* SOLR-10864: Simplified how Trie vs Points based numerics are randomized by

SolrTestCaseJ4 by adding a static
option to PointsField to ignore 'precisionStep' attribute. This change also
begins to attempt to randomize
'docValues' on numeric field types unless tests explicity enable/disable them.
(hossman, Steve Rowe)

* LUCENE-7883: Solr no longer uses the context class loader when resolving
resources, they are only resolved against Solr's own or "core" class loader
by default. (Uwe Schindler)

* SOLR-10915: Make builder based SolrClient constructors to be the only valid way
to construct client objects and
increase the visibility of builder elements to be protected so extending the
builder, and the clients is possible.
(Jason Gerlowski, Anshum Gupta)

* SOLR-10823: Add reporting period to SolrMetricReporter base class. (Christine

Poerschke)

* SOLR-10807: Randomize Points based numeric field types in all existing tests &
test schemas
where Trie numerics were previously hardcoded. (hossman, Steve Rowe, Anshum
Gupta)

* SOLR-6807: Changed requestDispatcher's handleSelect to default to false, thus

ignoring "qt".
 Simplified configs to not refer to handleSelect or "qt". Switch all tests that
assumed true to assume false
(added leading '/' in request handlers). Switch all tests referring to "standard"
request handler to
instead refer to "/select" with SearchHandler. Deprecated the old
StandardRequestHandler. (David Smiley)

* SOLR-10456: Deprecate timeout related setters from SolrClients, and replace with
Builder based implementation
(Jason Gerlowski, Anshum Gupta)

* SOLR-11004: Consolidate SolrClient builder code in an abstract base class (Jason

Gerlowski, Anshum Gupta)

* SOLR-10967: Cleanup the default configset. The changes involve some documentation
corrections, removing the currency
field from the examples and a few dynamic fields (Varun Thacker)

* SOLR-11016: Fix TestCloudJSONFacetJoinDomain test-only bug (hossman)

* SOLR-11021: The elevate.xml config-file is made optional in the

ElevationComponent.
The default configset doesn't ship with a elevate.xml file anymore (Varun
Thacker)

* SOLR-10898: Fix SOLR-10898 to not deterministicly fail 1/512 runs (hossman)

* SOLR-10796: TestPointFields: increase randomized testing of non-trivial values.

(Steve Rowe)

* SOLR-11068: MOVEREPLICA and REPLACENODE API parameter names are now 'sourceNode'
and 'targetNode'. The old names
viz. 'fromNode' for MOVEREPLICA and 'source', 'target' for REPLACENODE have been
deprecated. (shalin)

* SOLR-11088: Fix sporadic failures of MetricsHandlerTest.testPropertyFilter on

jenkins (shalin)

* SOLR-11037: Refactor to provide NodeConfig.getSolrDataHome internal API. (ab,

janhoy, shalin)

* SOLR-11119: Switch from Trie to Points field types in the .system collection
schema. (Steve Rowe)

* SOLR-10494: Make default response format JSON (wt=json), and also indent text
responses formats
(indent=on) by default (Trey Grainger & Cassandra Targett via hossman)

* SOLR-10760,SOLR-11315,SOLR-11313: Remove trie field types and fields from example

schemas. (Steve Rowe)

* SOLR-11056: Add random range query test that compares results across Trie*,
*Point and DocValue-only fields
(Tomás Fernández Löbbe)

* SOLR-10926: Increase the odds of randomly choosing point fields in our

SolrTestCaseJ4 numeric type randomization.
(hossman, Steve Rowe)
* SOLR-10846: ExternalFileField/FloatFieldSource should throw a clear exception on
initialization with
a Points-based keyField, which is not supported. (hossman, Steve Rowe)

* SOLR-11155: /analysis/field and /analysis/document requests should support points

fields.
(Jason Gerlowski, Steve Rowe)

* SOLR-10756: Undeprecate ZkStateReader.updateClusterState(), mark as

@lucene.internal, and rename to
forciblyRefreshAllClusterStateSlow(). (hossman, shalin, Steve Rowe)

* SOLR-11036: Separately report disk space metrics for solr.data.home and core root
directory. (ab)

* SOLR-10919: ord & rord functions give confusing errors with PointFields.
(hossman, Steve Rowe)

* SOLR-10847: Provide a clear exception when attempting to use the terms component
with points fields.
(hossman, Steve Rowe)

* SOLR-9321: Remove deprecated methods of ClusterState. (Jason Gerlowski, ishan,

Cao Manh Dat)

* SOLR-10033: When attempting to facet with facet.mincount=0 over points fields,

raise mincount to 1
and log a warning. (Steve Rowe)

* SOLR-11178: Change error handling in AutoScalingHandler to be consistent w/ other

APIs (noble)

* SOLR-11023: Added EnumFieldType, a non-Trie-based version of EnumField, and

deprecated EnumField
in favor of EnumFieldType. (hossman, Steve Rowe)

* SOLR-10803: Mark all Trie/LegacyNumeric based fields @deprecated in Solr7.

(Steve Rowe)

* SOLR-10821: Ref guide documentation for Autoscaling (Noble Paul, Cassandra

Targett, shalin)

* SOLR-11239: A special value of -1 can be specified for 'maxShardsPerNode' to

denote that
there is no limit. The bin/solr script send maxShardsPerNode=-1 when creating
collections. The
use of maxShardsPerNode is not supported when a cluster policy is in effect or
when a
collection specific policy is specified during collection creation.
(Noble Paul, shalin)

* SOLR-11183: V2 APIs are now available at /api endpoint. (Ishan Chattopadhyaya)

* SOLR-10617: JDBCStream accepts columns of type TIME, DATE & TIMESTAMP as well as
CLOBs and decimal
numeric types (James Dyer)

* SOLR-10400: Replace (instanceof TrieFooField || instanceof FooPointField)

constructs with
 FieldType.getNumberType() or SchemaField.getSortField() where appropriate.
(hossman, Steve Rowe)

* SOLR-10438: Assign explicit useDocValuesAsStored values to all points field types

in
schema-point.xml/TestPointFields. (hossman, Steve Rowe)

* LUCENE-7705: Allow CharTokenizer-derived tokenizers and KeywordTokenizer to

configure the max token length.
(Amrit Sarkar via Erick Erickson)

* SOLR-10659: Remove ResponseBuilder.getSortSpec use in

SearchGroupShardResponseProcessor.
(Judith Silverman via Christine Poerschke)

* SOLR-10741: Factor out createSliceShardsStr method from

HttpShardHandler.prepDistributed.
(Domenico Fabio Marino via Christine Poerschke)

* SOLR-10790: Fix warnings in Assign.java and ReplicaAssigner.java classes.

(Christine Poerschke)

* SOLR-8437: Improve RAMDirectory details in sample solrconfig files (Mark Miller,

Varun Thacker)

* SOLR-8762: return child docs in DIH debug (Gopikannan Venugopalsamy via Mikhail
Khludnev)

* SOLR-10501: Test sortMissing{First,Last} with points fields. (Steve Rowe)

* SOLR-10761: Switch trie numeric/date fields to points in data-driven-enabled

example and test schemas.
(Steve Rowe)

* SOLR-10851: SolrClients should clarify expectations for solrServerUrl parameter

(Jason Gerlowski
via Tomás Fernández Löbbe)

* SOLR-10891: BBoxField should support point-based number sub-fields. (Steve Rowe)

* SOLR-10834: Fixed tests and test configs to stop using numeric uniqueKey fields
(hossman)

* SOLR-10883: Ref guide: Escape replacement substitutions, e.g. => to right arrow,
so that they are
rendered visibly in the PDF. Also add .adoc file checks to the top-level
validate target, including
for the invisible substitutions PDF problem. (Steve Rowe)

* SOLR-10503,SOLR-10502: Deprecate CurrencyField in favor of new CurrencyFieldType,

which works
with point fields and provides control over dynamic fields used for the raw
amount and currency
code sub-fields. (hossman, Steve Rowe)

* SOLR-11261, SOLR-10966: Upgrade to Hadoop 2.7.4 to fix incompatibility with Java

9.
(Uwe Schindler)
* SOLR-11324: Clean up mention of trie fields in documentation and source comments.
(Steve Rowe)

================== 6.6.5 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.14.v20161028

New Features
----------------------

* SOLR-12530: Ability to disable configset upload via -

Dconfigset.upload.enabled=false startup parameter
(Ishan Chattopadhyaya)

Bug Fixes
----------------------

* SOLR-12450: Don't allow referal to external resources in various config files

(CVE-2018-8026).
(Yuyang Xiao, Uwe Schindler)

================== 6.6.4 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.17
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.11
Jetty 9.4.10.v20180503

Bug Fixes
----------------------

* SOLR-12316: Do not allow to use absolute URIs for including other files in
solrconfig.xml and schema parsing.
(Ananthesh, Ishan Chattopadhyaya, Uwe Schindler)

================== 6.6.3 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.17
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.11
Jetty 9.4.8.v20171121

Bug Fixes
----------------------

* SOLR-11971: Don't allow referal to external resources in DataImportHandler's

dataConfig request parameter.
(麦 香浓郁, Uwe Schindler)

* SOLR-11503: Collections created with legacyCloud=true cannot be opened if

legacyCloud=false (Erick Erickson)

* SOLR-11993: LeaderInitiatedRecoveryThread does not retry on UnknownHostException

(shalin, Cao Manh Dat, Steve Rowe)

================== 6.6.2 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.16
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.14.v20161028

Upgrade Notes
----------------------

* SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... })

the resolving of external entities is now disallowed by default.

* SOLR-11482: RunExecutableListener was deprecated and is disabled by default for

security reasons. Legacy applications still using it must explicitely pass
'-Dsolr.enableRunExecutableListener=true' to the Solr command line.
Be aware that you should really disable API-based config editing at the same
time, using '-Ddisable.configEdit=true'! (Uwe Schindler)

Bug Fixes
----------------------

* SOLR-11477: Disallow resolving of external entities in the XML query parser

(defType=xmlparser).
(Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke)

* SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is
trying to start some cores twice.
(Luiz Armesto, Shawn Heisey, Erick Erickson)

Other Changes
----------------------

* SOLR-10335: Upgrade to Tika 1.16. (Tim Allison, Tomás Fernández Löbbe via shalin)

================== 6.6.1 ==================

Bug Fixes
----------------------

* SOLR-10857: standalone Solr loads UNLOADed core on request (Erick Erickson,

Mikhail Khludnev)

* SOLR-11024: ParallelStream should set the StreamContext when constructing

SolrStreams (Joel Bernstein)

* SOLR-10908: CloudSolrStream.toExpression incorrectly handles fq clauses (Rohit

via Erick Erickson)

* SOLR-11177: CoreContainer.load needs to send lazily loaded core descriptors to

the proper list rather than send
them all to the transient lists. (Erick Erickson)

* SOLR-11122: Creating a core should write a core.properties file first and clean
up on failure
(Erick Erickson)

* SOLR-10910: Clean up a few details left over from pluggable transient core and
untangling
CoreDescriptor/CoreContainer references (Erick Erickson)

* SOLR-10721: Provide a way to know when Core Discovery is finished and when all
async cores are done loading
(Erick Erickson)

* SOLR-11069: CDCR bootstrapping can get into an infinite loop when a core is
reloaded (Amrit Sarkar, Erick Erickson)

* SOLR-11221: SolrJmxReporter broken on core reload. This resulted in some or most

metrics not being reported
via JMX after core reloads, depending on timing. (ab)

* SOLR-11261, SOLR-10966: Upgrade to Hadoop 2.7.4 to fix incompatibility with Java

9.
(Uwe Schindler)

* SOLR-11228: Exclude static html files in the partials directory from

authentication and authorization checks. The UI
will open correctly with kerberos enabled (Ishan Chattopadhyaya, Varun Thacker)

================== 6.6.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.10
Jetty 9.3.14.v20161028

Upgrade Notes
----------------------

* Solr contribs map-reduce, morphlines-core and morphlines-cell have been removed.

* JSON Facet API now uses hyper-log-log for numBuckets cardinality calculation and
calculates cardinality before filtering buckets by any mincount greater than 1.

* ZooKeeper dependency has been upgraded from 3.4.6 to 3.4.10.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-9992: Add support for grouping with PointFIelds. (Cao Manh Dat)

* SOLR-9994: Add support for CollapseQParser with PointFields. (Varun Thacker, Cao
Manh Dat)

* SOLR-10076: Hide keystore and truststore passwords from /admin/info/* outputs.

(Mano Kovacs via Mark Miller)

* SOLR-6736: Adding support for uploading zipped configsets using ConfigSets API
(Varun Rajput, Ishan Chattopadhyaya,
Noble Paul, Anshum Gupta, Gregory Chanan)

* SOLR-10349: Add totalTermFreq support to TermsComponent. (Shai Erera)

* SOLR-9993: Add support for ExpandComponent with PointFields. (Cao Manh Dat)

* SOLR-10239: MOVEREPLICA API (Cao Manh Dat, Noble Paul, shalin)

* SOLR-9936: Allow configuration for recoveryExecutor thread pool size. (Tim Owen
via Mark Miller)

* SOLR-10447: Collections API now supports a LISTALIASES command to return a list

of all collection aliases.
(Yago Riveiro, Ishan Chattopadhyaya, Mark Miller, Steve Molloy, Shawn Heisey,
Mike Drob, janhoy)

* SOLR-10446: CloudSolrClient can now be initialized using the base URL of a Solr
instance instead of
ZooKeeper hosts. This is possible through the use of newly introduced
HttpClusterStateProvider.
To fetch a list of collection aliases, this depends on LISTALIASES command, and
hence this way of
initializing CloudSolrClient would not work if you have collection aliases on
older versions of Solr
server that doesn't support LISTALIASES. (Ishan Chattopadhyaya, Noble Paul)

* SOLR-10082: Variance and Standard Deviation aggregators for the JSON Facet API.
Example: json.facet={x:"stddev(field1)", y:"variance(field2)"}
(Rustam Hashimov, yonik)
* SOLR-10505: Add multi-field support to TermsComponent when requesting terms'
statistics. (Shai Erera)

* SOLR-10537: SolrJ: Added SolrParams.toLocalParamsString() and

ClientUtils.encodeLocalParamVal. (David Smiley)

* SOLR-10507: Core Admin status command to emit collection details of each core
(noble)

* SOLR-10521: introducing sort=childfield(field) asc for searching by {!parent}

(Mikhail Khludnev)

* SOLR-9596: Add Solr support for SimpleTextCodec, via <codecFactory

class="solr.SimpleTextCodecFactory"/>
in solrconfig.xml (per-field specification in the schema is not possible). (Steve
Rowe)

* SOLR-1485: Add payload support with payload() value source and {!payload_score}
and {!payload_check}
query parsers. (Erik Hatcher)

* SOLR-10430: Add ls command to ZkCLI for listing only sub-directories. (Peter

Szantai-Kis via Mark Miller)

* SOLR-10583: JSON Faceting now supports a query time 'join' domain change option
(hossman)

* SOLR-9530: An Update Processor to convert normal update operation to an atomic

operations such as
add, set,inc, remove ,set, removeregex (Amrit Sarkar, noble)

* SOLR-10303: Add date/time Stream Evaluators (Gethin James, Dennis Gove, Joel
Bernstein)

* SOLR-10351: Add analyze Stream Evaluator to support streaming NLP (Joel

Bernstein)

* SOLR-10426: Add shuffle Streaming Expression (Joel Bernstein)

* SOLR-10274: The search Streaming Expression should work in non-SolrCloud mode

(Joel Bernstein)

* SOLR-10504: Add echo Streaming Expression (Joel Bernstein)

* SOLR-10516: Add eval Streaming Expression (Joel Bernstein)

* SOLR-10566: Add timeseries Streaming Expression (Joel Bernstein)

* SOLR-10559: Add let, get and tuple Streaming Expressions (Dennis Gove, Joel
Bernstein)

* SOLR-10582: Add Correlation Stream Evaluator (Joel Bernstein)

* SOLR-10536: stats Streaming Expression should work in non-SolrCloud mode (Joel

Bernstein)

* SOLR-10622: Add regress and predict Stream Evaluators (Joel Bernstein)

* SOLR-10626: Add covariance Stream Evaluator (Joel Bernstein)

* SOLR-10625: Add convolution Stream Evaluator (Joel Bernstein)

* SOLR-10638: Add normalize Stream Evaluator (Joel Bernstein)

* SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd.

(Ishan Chattopadhyaya, janhoy,
Noble Paul, Hrishikesh Gadre)

* SOLR-10292: Adds CartesianProductStream which turns a single tuple with a multi-

valued field into N
tuples, one for each value in the multi-valued field. (Dennis Gove)

Optimizations
----------------------

* SOLR-9184: Add a static convenience method ModifiableSolrParams#of(SolrParams)

which returns the same
instance if it already is modifiable, otherwise creates a new
ModifiableSolrParams instance.
(Jörg Rathlev via Koji)

* SOLR-10499: facet.heatmap is now significantly faster when the docset (base

query) matches everything and there are no
deleted docs. It's also faster when the docset matches a small fraction of the
index or none. (David Smiley)

* SOLR-9217: Reduced heap consumption for filter({!join ... score=...})

(Andrey Kudryavtsev, Gopikannan Venugopalsamy via Mikhail Khludnev)

* SOLR-10548: JSON Facet API now uses hyper-log-log++ for determining the number of
buckets
when merging requests from a multi-shard distributed request. (yonik)

* SOLR-10524: Better ZkStateWriter batching (Cao Manh Dat, Noble Paul, shalin,
Scott Blum)

* SOLR-10619: Optimize using cache for DistributedQueue in case of single-consumer

(Cao Manh Dat, Scott Blum)

Bug Fixes
----------------------
* SOLR-10281: ADMIN_PATHS is duplicated in two places and inconsistent. This can
cause automatic
retries to /admin/metrics handler by the CloudSolrClient. (shalin)

* SOLR-10108: bin/solr script recursive copy broken (Erick Erickson)

* SOLR-10362: "Memory Pool not found" error when reporting JVM metrics. (ab)

* SOLR-10369: bin\solr.cmd delete and healthcheck now works again; fixed

continuation chars ^ (Luis Goes via janhoy)

* SOLR-10387: zkTransfer normalizes destination path incorrectly if source is a

windows directory
(gopikannan venugopalsamy, Erick Erickson)

* SOLR-10323: fix to SpellingQueryConverter to properly strip out colons in field-

specific queries.
(Amrit Sarkar via James Dyer)

* SOLR-10264: Fixes multi-term synonym parsing in ManagedSynonymFilterFactory.

(Jörg Rathlev, Steve Rowe, Christine Poerschke)

* SOLR-8807: fix Spellcheck "collateMaxCollectDocs" parameter to work with queries

that have the
CollpasingQParserPlugin applied. (James Dyer)

* SOLR-10474: TestPointFields.testPointFieldReturn() depends on order of unsorted

hits. (Steve Rowe)

* SOLR-10473: Correct LBHttpSolrClient's confusing SolrServerException message when

timeAllowed is exceeded.
(Christine Poerschke)

* SOLR-10047: Mismatched Docvalues segments cause exception in Sorting/Faceting.

Solr now uninverts per segment
to avoid such exceptions. (Keith Laban via shalin)

* SOLR-10472: Fixed uninversion (aka: FieldCache) bugs with the numeric PointField
classes, and CurrencyField (hossman)

* SOLR-5127: Multiple highlight fields and wildcards are now supported e.g.
hl.fl=title,text_*
(Sven-S. Porst, Daniel Debray, Simon Endele, Christine Poerschke)

* SOLR-10493: Investigate SolrCloudExampleTest failures. (Erick Erickson)

* SOLR-10552: JSON Facet API numBuckets was not consistent between distributed and
non-distributed requests
when there was a mincount > 1. This has been corrected by changing numBuckets
cardinality processing to
ignore mincount > 1 for non-distributed requests. (yonik)

* SOLR-10520: child.facet.field doubled counts at least when rows>0. (Dr. Oleg

Savrasov via Mikhail Khludnev)

* SOLR-10480: Full pagination in JSON Facet API using offset does not work. (yonik)

* SOLR-10526: facet.heatmap didn't honor facet exclusions ('ex') for distributed

search. (David Smiley)

* SOLR-10500: nested child docs are adopted by neighbour when several parents come
in update/json/docs
(Alexey Suprun,noble via Mikhail Khludnev)

* SOLR-10316: Unloading a core can remove a ZK SolrCore registration entry for the
wrong SolrCore. (Mark Miller)

* SOLR-10588: Prevent redundant core reload on config update (Mikhail Khludnev)

* SOLR-10549: The new 'large' attribute had been forgotten in /schema/fieldtypes?

showDefaults=true (David Smiley)

* SOLR-10615: requests are suspended until SolrDispatchFilter initialization is

completed.
After core container shutdown or severe initialization problem Solr responds with
 http stauts 404 Not Found instead of 500 as it was before (Mikhail Khludnev)

* SOLR-8149: Admin UI - Plugins / Stats - active item is now highlighted (Labuzov

Dmitriy via janhoy)

* SOLR-10630: HttpSolrCall.getAuthCtx().new AuthorizationContext()

{...}.getParams()
sometimes throws java.lang.NullPointerException (hu xiaodong via shalin)

* SOLR-9527: Improve distribution of replicas when restoring a collection

(Hrishikesh Gadre, Stephen Lewis, Rohit, Varun Thacker)

* LUCENE-7821: The classic and flexible query parsers, as well as Solr's

"lucene"/standard query parser, should require " TO " in range queries,
and accept "TO" as endpoints in range queries. (hossman, Steve Rowe)

* SOLR-10735: Windows script (solr.cmd) didn't work properly with directory

containing spaces. Adding quotations
to fix (Uwe Schindler, janhoy, Tomas Fernandez-Lobbe, Ishan Chattopadhyaya)

Ref Guide
----------------------

* SOLR-10758: Modernize the Solr ref guide's Chinese language analysis coverage.
(Steve Rowe)

Other Changes
----------------------

* SOLR-9221: Remove Solr contribs: map-reduce, morphlines-core and morphlines-cell.

(Steve Rowe)

* SOLR-10249: Refactor IndexFetcher.doFetch() to return a more detailed result.

(Jeff Miller via David Smiley)

* SOLR-10304: Refactor Document handling out of SolrIndexSearcher into a new class

"SolrDocumentFetcher".
Deprecated SolrPluginUtils.docListToSolrDocumentList(). (David Smiley)

* SOLR-10352: bin/solr script now prints warning when available system entropy is
lower than 300 (Esther Quansah via
Ishan Chattopadhyaya)

* SOLR-10344: Update Solr default/example and test configs to use

WordDelimiterGraphFilterFactory. (Steve Rowe)

* SOLR-10343: Update Solr default/example and test configs to use

SynonymGraphFilterFactory. (Steve Rowe)

* SOLR-10365: Handle a SolrCoreInitializationException while publishing core state

during SolrCore creation
(Ishan Chattopadhyaya)

* SOLR-10357: Enable edismax and standard query parsers to handle the option
combination
sow=false / autoGeneratePhraseQueries="true" by setting
QueryBuilder.autoGenerateMultiTermSynonymsQuery.
(Steve Rowe)
* SOLR-10147: Admin UI -> Cloud -> Graph: Impossible to see shard state (Amrit
Sarkar, janhoy)

* SOLR-10399: Generalize some internal facet logic to simplify points/non-points

field handling (Adrien Grand, hossman)

* SOLR-7383: New DataImportHandler 'atom' example, replacing broken 'rss' example

(Alexandre Rafalovitch)

* SOLR-9601: Redone DataImportHandler 'tika' example, removing all unused and

irrelevant definitions (Alexandre Rafalovitch)

* SOLR-8906: Make transient core cache pluggable (Erick Erickson)

* SOLR-9745: print errors from solr.cmd (Gopikannan Venugopalsamy via Mikhail

Khludnev)

* SOLR-10394: Rename getSortWithinGroup to getWithinGroupSort in

search.grouping.Command class.
(Judith Silverman, Christine Poerschke)

* SOLR-10440: LBHttpSolrClient.doRequest is now always wrapped in a Mapped

Diagnostic Context (MDC).
(Christine Poerschke)

* SOLR-10429: UpdateRequest#getRoutes()should copy the response parser (noble)

* SOLR-10007: Clean up references to CoreContainer and CoreDescriptors (Erick

Erickson)

* SOLR-10151: Use monotonically incrementing counter for doc ids in TestRecovery.

(Peter Szantai-Kis, Mano Kovacs via Mark Miller)

* SOLR-10514: Upgrade Metrics library to 3.2.2. (ab)

* SOLR-9386: Upgrade Zookeeper to 3.4.10. (Shawn Heisey, Steve Rowe)

* SOLR-10519: SolrCLI.atPath cannot handle children that begin with a slash. (Erick
Erickson)

* SOLR-9867: Adding isLoading=true as core status. Fixing start after stop scenario
in bin/solr
(Andrey Kudryavtsev, Mikhail Khludnev)

* SOLR-7041: Cutover tests to using 'q.op' and 'df' instead of schema

'defaultOperator' and 'defaultSearchField' (janhoy)

* SOLR-10601: StreamExpressionParser should handle white space around = in named

parameters (Joel Bernstein)

* SOLR-10614: Static fields have turned to instance's field in SimplePostTool.

Enabled TestSolrCLIRunExample.testTechproductsExample(). (Andrey Kudryavtsev,
Mikhail Khludnev)

* SOLR-10522: Revert SpellCheckComponent response format change from SOLR-9972

(rel. 6.5.0). While this
was an improvement for the json "arrntv" format, it caused problems for the
default json format.
(James Dyer, reported by Nikita Pchelintsev)
* SOLR-10644: solr.in.sh installed by install script should be writable by solr
user (janhoy)

* SOLR-10729: Deprecated LatLonType, GeoHashField, SpatialPointVectorFieldType, and

SpatialTermQueryPrefixTreeFieldType.
Instead, switch to LatLonPointSpatialField or SpatialRecursivePrefixTreeFieldType
or RptWithGeometrySpatialField.
(David Smiley)

================== 6.5.1 ==================

Bug Fixes
----------------------

* SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr

OriginalScoreFeature class.
(Vitezslav Zak, Christine Poerschke)

* SOLR-10416: The JSON output of /admin/metrics is fixed to write the container as

a
map (SimpleOrderedMap) instead of an array (NamedList). (shalin)

* SOLR-10277: On 'downnode', lots of wasteful mutations are done to ZK.

(Joshua Humphries, Scott Blum, Varun Thacker, shalin)

* SOLR-10421: Fix params persistence for solr/contrib/ltr (MinMax|

Standard)Normalizer classes.
(Jianxiong Dong, Christine Poerschke)

* SOLR-10404: The fetch() streaming expression wouldn't work if a value included

query syntax chars (like :+-).
Fixed, and enhanced the generated query to not pollute the queryCache. (David
Smiley)

* SOLR-10423: Disable graph query production via schema configuration

<fieldtype ... enableGraphQueries="false">.
This fixes broken queries for ShingleFilter-containing query-time analyzers when
request param sow=false.
(Steve Rowe)

* SOLR-10425: Fix indexed="false" on numeric PointFields (Tomás Fernández Löbbe,

hossman)

* SOLR-10341: SQL AVG function mis-interprets field type. (Joel Bernstein)

* SOLR-10444: SQL interface does not use client cache. (Joel Bernstein)

* SOLR-10420: Solr 6.x leaking one SolrZkClient instance per second (Scott Blum,
Cao Manh Dat, Markus Jelsma, Steve Rowe)

* SOLR-10439: The new 'large' attribute had been forgotten in /schema/fields?

showDefaults=true (David Smiley)

* SOLR-10527: edismax with sow=false fails to create dismax-per-term queries when

any field is boosted.
(Steve Rowe)

================== 6.5.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.14.v20161028
Apache Calcite 1.11.0

Detailed Change List

----------------------

Upgrade Notes
----------------------
* SOLR-10226: JMX metric "avgTimePerRequest" (and the corresponding metric in the
metrics API for
each handler) used to be a simple non-decaying average based on total cumulative
time and the
number of requests. New Codahale Metrics implementation applies exponential decay
to this value,
which heavily biases the average towards the last 5 minutes. (ab)

* SOLR-8593: Parallel SQL now uses Apache Calcite as its SQL framework. As part of
this change
the default aggregation mode has been changed to facet rather than map_reduce.
There has also been changes
to the SQL aggregate response and some SQL syntax changes. Consult the
documentation for full details.

New Features
----------------------

* SOLR-9836: Add ability to recover from leader when index corruption is detected
on SolrCore creation.
(Mike Drob via Mark Miller)

* SOLR-9926: Allow passing arbitrary java system properties to zkcli. (Hrishikesh

Gadre via Mark Miller)

* SOLR-9885: Allow pre-startup Solr log management in Solr bin scripts to be

disabled. (Mano Kovacs via Mark Miller)

* SOLR-9481: Authentication and Authorization plugins now work in standalone mode

if security.json is placed in
SOLR_HOME on every node. Editing config through API is supported but affects only
that one node. (janhoy)

* SOLR-8029: Added new style APIs and a framework for creating new APIs and mapping
old APIs to new
(noble, Steve Rowe, Cassandra Targett, Timothy Potter)

* SOLR-9933: SolrCoreParser now supports configuration of custom SpanQueryBuilder

classes.
 (Daniel Collins, Christine Poerschke)

* SOLR-7955: Auto create .system collection on first request if it does not exist
(noble)

* SOLR-10087: StreamHandler now supports registering custom streaming expressions

from the blob store (Kevin Risden)

* SOLR-9997: Enable configuring SolrHttpClientBuilder via java system property.

(Hrishikesh Gadre via Mark Miller)

* SOLR-9912: Add facet.excludeTerms parameter support. (Jonny Marks, David Smiley,

Christine Poerschke)

* SOLR-9916: Adds Stream Evaluators to support evaluating values from tuples.

Supports boolean,
numeric, and conditional evaluators. BooleanOperations have been removed in
preference of
BooleanEvaluators. (Dennis Gove)

* SOLR-9903: Stop interrupting the update executor on shutdown, it can cause

graceful shutdowns to put replicas into Leader
Initiated Recovery among other undesirable things. (Mark Miller)

* SOLR-8396: Add support for PointFields in Solr (Ishan Chattopadhyaya, Tomás

Fernández Löbbe)

* SOLR-9987: Add support for MultiValued DocValues in PointFields using

SortedNumericDocValues (Tomás Fernández Löbbe)

* SOLR-5944: In-place updates of Numeric DocValues. To leverage this, the _version_

field and the updated
field must both be stored=false, indexed=false, docValues=true. (Ishan
Chattopadhyaya, hossman, noble,
shalin, yonik)

* SOLR-10158: Add support for "preload" option in MMapDirectoryFactory.

(Amrit Sarkar via Uwe Schindler)

* SOLR-10153 & SOLR-10152: The Unified and Postings based highlighters: Add
hl.bs.type=SEPARATOR along with new param
hl.bs.separator to break passages by a provided single character. (Amrit Sarkar,
David Smiley)

* SOLR-10156: Add significantTerms Streaming Expression (Joel Bernstein)

* SOLR-8593: Integrate Apache Calcite into the SQLHandler (Kevin Risden, Cao Manh
Dat, Joel Bernstein)

* SOLR-10146: Added button to the Admin UI 'Collection' tab for deleting an

inactive shard (Amrit Sarkar, janhoy)

* SOLR-9999: Instrument DirectUpdateHandler2. This registers existing statistics

under metrics API and adds
more metrics to track the rates of update and delete commands. (ab)

* SOLR-9986: Implement DatePointField (Cao Manh Dat, Tomás Fernández Löbbe)

* SOLR-8045: Deploy V2 API at /v2 instead of /solr/v2 (Cao Manh Dat, Noble Paul)
* SOLR-10039: New LatLonPointSpatialField replacement for LatLonType (and some uses
of RPT). Multi-value capable
indexed geo lat-lon points, query by rect or circle. Efficient distance
sorting/boosting too. (David Smiley)

* SOLR-10250: CloudSolrClient can now return versions for documents added or

deleted when "versions=true" is passed.
However, if there is a leader election while this request is in transit, the
versions may not be returned from that
shard. (Boris Naguet, Ishan Chattopadhyaya)

* SOLR-9045: Make RecoveryStrategy settings configurable. (Christine Poerschke)

* SOLR-10224: Add disk total and disk free metrics. (ab)

* SOLR-10085: SQL result set fields should be ordered by the field list (Joel
Bernstein)

* SOLR-10254: significantTerms Streaming Expression should work in non-SolrCloud

mode (Joel Bernstein)

* SOLR-10286: string/text fields may now declare themselves as large="true" in the

schema. Large fields are always
lazy loaded and will only take up space in the document cache if the actual value
is < 512KB. This option
requires "stored" and must not be multiValued. It's intended for fields that
might have very large values so that
they don't get cached in memory. (David Smiley)

* SOLR-9185: Solr's edismax and "Lucene"/standard query parsers will no longer

split on whitespace before sending
terms to analysis, if given the "sow=false" request param ("sow"=>"split on
whitespace"). This enables multi-term
source synonyms to match at query-time using SynonymGraphFilterFactory; other
analysis components will also now
work at query time, e.g. ShingleFilterFactory. By default, and when the
"sow=true" param is specified, these
parsers' behavior remains the same: queries will be split on whitespace before
sending individual terms to analysis.
(Steve Rowe)

Bug Fixes
----------------------

* SOLR-9976: Fix init bug in SegmentsInfoRequestHandlerTest (hossman)

* SOLR-9977: Fix config bug in DistribDocExpirationUpdateProcessorTest that allowed

false assumptions
about when index version changes (hossman)

* SOLR-9979: Macro expansion should not be done in shard requests (Tomás Fernández
Löbbe)

* SOLR-9114: NPE using TermVectorComponent, MoreLikeThisComponent in combination

with ExactStatsCache (Cao Manh Dat, Varun Thacker)

* SOLR-10049: Collection deletion leaves behind the snapshot metadata (Hrishikesh

Gadre via yonik)
* SOLR-10120: A SolrCore reload can remove the index from the previous SolrCore
during replication index rollover. (Mark Miller)

* SOLR-10124: Replication can skip removing a temporary index directory in some

cases when it should not. (Mark Miller)

* SOLR-9996: Unstored IntPointField returns Long type (Ishan Chattopadhyaya)

* SOLR-10104: BlockDirectoryCache release hooks do not work with multiple

directories. (Mike Drob, Mark Miller)

* SOLR-10121: Fix race conditions in HDFS BlockCache that can contribute to

corruption in high
concurrency situations. (yonik)

* SOLR-10141: Upgrade to Caffeine 2.4.0 since v1.0.1 contributed to BlockCache

corruption because the
removal listener was called more than once for some items and not at all for
other items. (Ben Manes, yonik)

* SOLR-10114: Reordered delete-by-query causes inconsistenties between shards that

have
child documents (Mano Kovacs, Mihaly Toth, yonik)

* SOLR-10159: When DBQ is reordered with an in-place update, upon whose updated
value the DBQ is based
on, the DBQ fails due to excessive caching in DeleteByQueryWrapper (Ishan
Chattopadhyaya)

* SOLR-10020: CoreAdminHandler silently swallows some errors. (Mike Drob via Erick
Erickson)

* SOLR-10063: CoreContainer shutdown has race condition that can cause a hang on
shutdown. (Mark Miller)

* SOLR-10170: ClassCastException in RecoveryStrategy. (Mark Miller)

* SOLR-9846: Overseer is not always closed after being started. (Mark Miller)

* SOLR-10168: ShardSplit can fail with NPE in

OverseerCollectionMessageHandler#waitForCoreAdminAsyncCallToComplete. (Mark Miller)

* SOLR-9824: Some bulk update paths could be very slow due to CUSC polling. (David
Smiley, Mark Miller)

* SOLR-10055: Linux installer now renames existing bin/solr.in.* as

bin/solr.in.*.orig to make the installed config in
/etc/defaults be the one found by default when launching solr manually. (janhoy)

* SOLR-10196: ElectionContext#runLeaderProcess can hit NPE on core close. (Mark

Miller)

* SOLR-10225: Fix HDFS BlockCache evictions metric to not count explicit removal
due to a directory close. (yonik)

* SOLR-10088: Installer script does not put zoo.cfg in SOLR_HOME (janhoy)

* SOLR-10226: add back "totalTime" metric to all handlers. See also the back-compat
note. (ab)

* SOLR-9838: "inc" atomic update doesn't respect default field value (hoss, Amrit
Sarkar, Ishan Chattopadhyaya)

* SOLR-10269: MetricsHandler JSON output incorrect. (ab)

* SOLR-10279: The autoAddReplica feature can result in SolrCores being assigned new
shards when using
legacyCloud=false and will also fail on a state check when taking over a core
registration with a new
core. (Mark Miller, Hrishikesh Gadre, Patrick Dvorack)

* SOLR-10184: Fix bin/solr so it can run properly on java9 (hossman, Uwe Schindler)

* SOLR-9516: Admin UI (angular) now works with Kerberos, by excluding serving of

/solr/libs/* through
SolrDispatchFilter. (Cassandra Targett, Amrit Sarkar via Ishan Chattopadhyaya)

* SOLR-10302: Solr's zkcli scripts now able to find the metrics libraries, which it
couldn't earlier (kiran, Ishan Chattopadhyaya)

* SOLR-10283: Learning to Rank (LTR) SolrFeature to reject searches with missing

efi (External Feature Information) used by fq.
(Christine Poerschke)

* SOLR-10237: Poly-fields should work with subfields that have docValues=true

(Tomás Fernández Löbbe, David Smiley)

* SOLR-10218: The Schema API commands "add-field-type" and "replace-field-type"

improperly specify SimilarityFactory params.
(Benjamin Deininger, Troy Mohl, Steve Rowe)

* SOLR-10319: SolrCore "instanceDir" metric not visible in JMX. (ab)

Optimizations
----------------------

* SOLR-9941: Clear the deletes lists at UpdateLog before replaying from log. This
prevents redundantly pre-applying
DBQs, during the log replay, to every update in the log as if the DBQs were out
of order. (hossman, Ishan Chattopadhyaya)

* SOLR-9764: All filters that match all documents in the index now share the same
memory (DocSet).
(Michael Sun, yonik)

* SOLR-9584: Support Solr being proxied with another endpoint than default /solr,
by using relative links
in AdminUI javascripts (Yun Jie Zhou via janhoy)

* SOLR-10143: PointFields will create IndexOrDocValuesQuery when a field is both,

indexed=true and docValues=true
(Tomás Fernández Löbbe)

* SOLR-10273: The field with the longest value (if it exceeds 4K) is moved to be
last in the Lucene Document in order
to benefit from stored field optimizations in Lucene that can avoid reading it
when it's not needed. If the field is
 multi-valued, they all move together to the end to retain order. (David Smiley)

Other Changes
----------------------
* SOLR-9980: Expose configVersion in core admin status (Jessica Cheng Mallet via
Tomás Fernández Löbbe)

* SOLR-9972: SpellCheckComponent collations and suggestions returned as a JSON

object rather than a list
(Christine Poerschke in response to bug report from Ricky Oktavianus Lazuardy)

* SOLR-9983: Fixing NullPointerException failure by TestManagedSchemaThreadSafety

adding check for Zookeeper session expiration (Steve Rowe, Mikhail Khludnev)

* SOLR-10043: Reduce logging of pre-start log rotation (janhoy)

* SOLR-10018: Increase the default hl.maxAnalyzedChars to 51200 for the Unified &
Postings Highlighter so that all
highlighters now have this same default. (David Smiley)

* SOLR-6246: Added tests to check that the changes in LUCENE-7564 and LUCENE-7670
enable AnalyzingInfixSuggester and BlendedInfixSuggester to play nicely with core
reload.
SolrSuggester.build() now throws SolrCoreState.CoreIsClosedException when
interrupted
by a core reload/shutdown. (Steve Rowe)

* SOLR-9800: Factor out FacetComponent.newSimpleFacets method. (Jonny Marks via

Christine Poerschke)

* SOLR-9914: SimpleFacets: refactor "contains" check into "SubstringBytesRefFilter"

class.
(Jonny Marks, David Smiley, Christine Poerschke)

* SOLR-10072: The test TestSelectiveWeightCreation appears to be unreliable.

(Michael Nilsson via Mark Miller)

* SOLR-10011: Refactor PointField & TrieField to now have a common base class,
NumericFieldType. The
TrieField.TrieTypes and PointField.PointTypes are now consolidated to
NumericFieldType.NumberType. This
refactoring also fixes a bug whereby PointFields were not using DocValues for
range queries for
indexed=false, docValues=true fields. (Ishan Chattopadhyaya, Tomás Fernández
Löbbe)

* SOLR-9890: factor out ShardResultTransformerUtils.[un]marshSortValue methods

(Judith Silverman, Christine Poerschke)

* SOLR-9966: Convert/migrate tests using EasyMock to Mockito (Cao Manh Dat, Uwe
Schindler)

* SOLR-10173: Make HttpShardHandlerFactory.getReplicaListTransformer more

extensible.
(Ramsey Haddad via Christine Poerschke)

* SOLR-9842: UpdateRequestProcessors have no way to guarantee the closing of

resources used for a request.
(Mark Miller)
* SOLR-9848: Lower solr.cloud.wait-for-updates-with-stale-state-pause back down
from 7 seconds.
(Mark Miller)

* SOLR-10020: Cannot reload a core if it fails initialization. (Mike Drob via Erick
Erickson)

* SOLR-9450: The docs/ folder in the binary distribution now contains a single
index.html file linking
to the online documentation, reducing the size of the download (janhoy, Shawn
Heisey, Uwe Schindler)

* SOLR-7453: Remove replication & backup scripts in the solr/scripts directory of

the checkout (Varun Thacker)

* SOLR-10214: Remove unused HDFS BlockCache metrics and add storeFails, as well as
adding total
counts for lookups, hits, and evictions. (yonik)

* SOLR-10134: EmbeddedSolrServer responds on Schema API requests (Robert

Alexandersson via Mikhail Khludnev)

* SOLR-10219: re-enable HDFS tests under JDK9 (hossman, Uwe Schindler)

* SOLR-10155: For numeric types facet.contains= and facet.prefix= are now rejected.
(Gus Heck, Christine Poerschke)

* SOLR-10171: Add Constant Reduction Rules to Calcite Planner (Kevin Risden)

* SOLR-10230: default TTL of PKIAuthenticationPlugin increased to 10secs (noble)

* SOLR-10235: Fix DIH's TestJdbcDataSource to work with Java 9 and other Java
runtimes that
do not use the same DriverManager implementation like Oracle's original one. The
test now
uses a fully implemented Driver instance returning a mock connection. The test
also works
correct now if other drivers were installed before test execution (e.g., through
IDE).
(hossman, Uwe Schindler)

* SOLR-8876: change morphline test config files to work around 'importCommands' bug
when using java9 (hossman)

* SOLR-10247: Support non-numeric metrics and a "compact" format of /admin/metrics

output. (ab)

* SOLR-9990: Add PointFields in example/default schemas (Tomás Fernández Löbbe)

================== 6.4.2 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.14.v20161028

Detailed Change List

----------------------

Bug Fixes
----------------------
* SOLR-10130, SOLR-10182: Serious performance degradation in Solr 6.4.1 due to the
new metrics collection.
Default settings in solrconfig.xml /config/indexConfig/metrics have been changed
to turn off
IndexWriter metrics collection. Directory level metrics collection has been
completely removed until
a better design is found. (ab, ishan)

* SOLR-10138: Transaction log replay can hit an NPE due to new Metrics code. (ab)

* SOLR-10083: Fix instanceof check in ConstDoubleSource.equals (Pushkar Raste via

Christine Poerschke)

* SOLR-10190: Fix NPE in CloudSolrClient when reading stale alias (Janosch Woschitz
via Tomás Fernández Löbbe)

* SOLR-10192: Fix copy/paste in solr-ltr pom.xml template. (Christine Poerschke)

================== 6.4.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.14.v20161028

Bug Fixes
----------------------
* SOLR-9969: "Plugin/Stats" section of the UI doesn't display empty metric types
(Tomás Fernández Löbbe)

* SOLR-8491: solr.cmd SOLR_SSL_OPTS is overwritten (Sam Yi, Andy Hind, Marcel

Berteler, Kevin Risden)

* SOLR-10031: Validation of filename params in ReplicationHandler (Hrishikesh

Gadre, janhoy)

* SOLR-10054: Core swapping doesn't work with new metrics changes in place (ab)

================== 6.4.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.
Versions of Major Components
---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.14.v20161028

Detailed Change List

----------------------

Upgrade Notes
----------------------

* SOLR-9166: Export handler returns zero for numeric fields that are not in the
original doc. One
consequence of this change is that you must be aware that some tuples will not
have values if
there were none in the original document.

* SOLR-8785: Metrics related classes in org.apache.solr.util.stats have been

removed in favor of
the dropwizard metrics library. Any custom plugins using these classes should be
changed to use
the equivalent classes from the metrics library.
As part of this, the following changes were made to the output of Overseer Status
API:
* The "totalTime" metric has been removed because it is no longer supported
* The metrics "75thPctlRequestTime", "95thPctlRequestTime", "99thPctlRequestTime"
and "999thPctlRequestTime" in Overseer Status API have been renamed to
"75thPcRequestTime", "95thPcRequestTime"
and so on for consistency with stats output in other parts of Solr.
* The metrics "avgRequestsPerMinute", "5minRateRequestsPerMinute" and
"15minRateRequestsPerMinute" have been
replaced by corresponding per-second rates viz. "avgRequestsPerSecond",
"5minRateRequestsPerSecond"
and "15minRateRequestsPerSecond" for consistency with stats output in other
parts of Solr.

* SOLR-9708: You are encouraged to try out the UnifiedHighlighter by setting

hl.method=unified and report feedback. It
might become the default in 7.0. It's more efficient/faster than the other
highlighters, especially compared to the
original Highlighter. That said, some options aren't supported yet.
It will get more features in time, especially with your input. See
HighlightParams.java
for a listing of highlight parameters annotated with which highlighters use them.
hl.useFastVectorHighlighter is now considered deprecated in lieu of
hl.method=fastVector.

* SOLR-9712: maxWarmingSearchers now defaults to 1, and more importantly commits

will now block if this
limit is exceeded instead of throwing an exception (a good thing). Consequently
there is no longer a
risk in overlapping commits. Nonetheless users should continue to avoid
excessive committing. Users are
advised to remove any pre-existing maxWarmingSearchers entries from their
solrconfig.xml files.

* SOLR-7466: complexphrase query parser now supports leading wildcards, beware of

its' possible heaviness.
Users are encouraged to use ReversedWildcardFilter in index time analysis.

New Features
----------------------
* SOLR-9918: Add SkipExistingDocumentsProcessor that skips duplicate inserts and
ignores updates to missing docs
(Tim Owen via koji)

* SOLR-9293: Solrj client support for hierarchical clusters and other topics
marker. (Dawid Weiss)

* SOLR-9681: FacetModule / JSON Facet API added the ability to add filters directly
to
any facet command. The filters are applied after any domain change operations.
Example: { type:terms, field:category, filter:"user:yonik" }
(yonik)

* SOLR-9442, SOLR-9787: Adds Array of Name Type Value (json.nl=arrntv) style to

JSONResponseWriter.
(Jonny Marks, Christine Poerschke, hossman)

* SOLR-8542: Adds Solr Learning to Rank (LTR) plugin for reranking results with
machine learning models.
(Michael Nilsson, Diego Ceccarelli, Joshua Pantony, Jon Dorando, Naveen
Santhapuri, Alessandro Benedetti, David Grohmann, Christine Poerschke)

* SOLR-9055: Make collection backup/restore extensible. (Hrishikesh Gadre, Varun

Thacker, Mark Miller)

* SOLR-9682: JSON Facet API: added "param" query type to facet domain filter
specification to obtain
filters via query parameters. (yonik)

* SOLR-9038: Add a command-line tool to manage the snapshots functionality

(Hrishikesh Gadre via yonik)

* SOLR-9633: Limit memory consumed by FastLRUCache with a new 'maxRamMB' config

parameter.
(yonik, Michael Sun, shalin)

* SOLR-9666: SolrJ LukeResponse support dynamic fields (Fengtan via Kevin Risden)

* SOLR-9077: Streaming expressions should support collection alias (Kevin Risden)

* SOLR-9324: Support Secure Impersonation / Proxy User for solr authentication

(Gregory Chanan, Hrishikesh Gadre via yonik)

* SOLR-9721: javabin Tuple parser for streaming and other end points (noble)

* SOLR-9708: Added UnifiedSolrHighlighter, a highlighter adapter for Lucene's

UnifiedHighlighter. The adapter is a
derivative of the PostingsSolrHighlighter, supporting mostly the same parameters
with some differences.
Introduced "hl.method" parameter which can be set to original|fastVector|
postings|unified to pick the highlighter at
 runtime without the need to modify solrconfig from the default configuration.
hl.useFastVectorHighlighter is now
considered deprecated in lieu of hl.method=fastVector. (Timothy Rodriguez, David
Smiley)

* SOLR-9728: Ability to specify Key Store type in solr.in.sh file for SSL (Michael
Suzuki, Kevin Risden)

* SOLR-5043: New solr.dns.prevent.reverse.lookup system property that can be used

to prevent long core
(re)load delays on systems with missconfigured hostname/DNS (hossman)

* SOLR-9844: FieldCache information fetched via the mbeans handler or seen via the
UI now displays the total size used.
The individual cache entries in the response are now formatted better as well.
(Varun Thacker)

* SOLR-9513: Generic authentication plugins (GenericHadoopAuthPlugin and

ConfigurableInternodeAuthHadoopPlugin) that delegate
all functionality to Hadoop authentication framework. (Hrishikesh Gadre via Ishan
Chattopadhyaya)

* SOLR-9860: Enable configuring invariantParams via HttpSolrClient.Builder

(Hrishikesh Gadre, Ishan Chattopadhyaya)

* SOLR-4735, SOLR-9921: Improve metrics reporting. This uses the dropwizard metric
library, adding an internal
API for registering and reporting metrics from Solr components. Several new
metrics and an improved JMX
reporter have been added (Alan Woodward, Jeff Wartes, Christine Poerschke, Kelvin
Wong, shalin, ab)

* SOLR-9788: Use instrumented jetty classes provided by the dropwizard metric

library. (shalin)

* SOLR-9805: Use metrics-jvm library to instrument jvm internals such as GC, memory
usage and others. (shalin)

* SOLR-9812, SOLR-9911, SOLR-9960: Added a new /admin/metrics API to return all

metrics collected by Solr via API.
API supports four optional multi-valued parameters:
- 'group' (all,jvm,jetty,node,core),
- 'type' (all,counter,timer,gauge,histogram),
- 'prefix' that filters the returned metrics,
- 'registry' that selects one or more registries by prefix (eg.
solr.jvm,solr.core.collection1)
- Example: http://localhost:8983/solr/admin/metrics?group=jvm,jetty&type=counter
- Example: http://localhost:8983/solr/admin/metrics?group=jvm&prefix=buffers,os
- Example: http://localhost:8983/solr/admin/metrics?
registry=solr.node,solr.core&prefix=ADMIN
(shalin, ab)

* SOLR-9884: Add version to segments handler output (Steven Bower via Erick
Erickson)

* SOLR-9725: Substitute properties into JdbcDataSource configuration ( Jamie

Jackson, Yuri Sashevsky via Mikhail Khludnev)

* SOLR-9877: SOLR-9923: SOLR-9948: Use instrumented http client and connection pool
in HttpShardHandler and
UpdateShardHandler. The metrics are collected per query-less URL and method by
default but it can be configured
to host/method and per-method as well. (shalin)

* SOLR-9880: Add Ganglia, Graphite and SLF4J metrics reporters. (ab)

* SOLR-9897: Add hl.requireFieldMatch toggle support when using the

UnifiedHighlighter. Defaults to false like the
other highlighters that support this. (David Smiley)

* SOLR-9905: Add NullStream to isolate the performance of the ExportWriter (Joel

Bernstein)

* SOLR-9891: Add mkroot command to bin/solr and bin/solr.cmd (Erick Erickson)

* SOLR-9668,SOLR-7197: introduce cursorMark='true' in SolrEntityProcessor (Yegor

Kozlov, Raveendra Yerraguntl via Mikhail Khludnev)

* SOLR-9684: Add priority Streaming Expression (Joel Bernstein, David Smiley)

* SOLR-9896: Instrument and collect metrics from query, update, core admin and core
load thread pools. (shalin)

* SOLR-9854: Collect metrics for index merges and index store IO (ab)

* SOLR-8530: Add HavingStream to Streaming API and StreamingExpressions (Joel

Bernstein)

* SOLR-7466: Enable leading wildcard in complexphrase query parser, optimize it

with ReversedWildcardFilterFactory
when it's provided (Mikhail Khludnev)

* SOLR-9935: Add hl.fragsize support when using the UnifiedHighlighter to avoid

snippets/Passages that are too small.
Defaults to 70. (David Smiley)

* SOLR-9856: Collect metrics for shard replication and tlog replay on replicas
(ab).

* SOLR-9886: Add a 'enable' flag to caches to enable/disable them (Pushkar Raste,

noble)

* SOLR-9947: Clean up some SolrInfoMBean categories. Add an alternative

hierarchical view in JMX
for SolrInfoMBeans, which uses similar conventions to SolrJmxReporter. (ab)

Optimizations
----------------------
* SOLR-9704: Facet Module / JSON Facet API: Optimize blockChildren facets that have
filters specified by using those filters as acceptDocs. (yonik)

* SOLR-9726: Reduce number of lookupOrd calls made by the DocValuesFacets.getCounts

method.
(Jonny Marks via Christine Poerschke)

* SOLR-9772: Deriving distributed sort values (fieldSortValues) should reuse

comparator and only invalidate leafComparator. (John Call via yonik)
* SOLR-9786: FieldType has a new getSetQuery() method that can take a set of terms
and create a more efficient query (such as TermsQuery). The solr query parser
has been
changed to use this method when appropriate. The parser also knows when it is
being
used to parse a filter and will create TermsQueries from large lists of normal
terms
or numbers, resulting in a query that will execute faster. This also acts to
avoid
BooleanQuery maximum clause limit. Query parsing itself has also been optimized,
resulting in less produced garbage and 5-7% better performance.
(yonik)

* SOLR-9902: StandardDirectoryFactory should use Files API for it's move

implementation. (Mark Miller, Mike Drob)

Bug Fixes
----------------------
* SOLR-9701: NPE in export handler when "fl" parameter is omitted.
(Erick Erickson)

* SOLR-9433: SolrCore clean-up logic uses incorrect path to delete dataDir on

failure to create a core.
(Evan Sayer, shalin)

* SOLR-9360: Solr script not properly checking SOLR_PID

(Alessandro Benedetti via Erick Erickson)

* SOLR-9716: RecoveryStrategy sends prep recovery command without setting read time
out which can cause
replica recovery to hang indefinitely on network partitions. (Cao Manh Dat,
shalin)

* SOLR-9624: In Admin UI, do not attempt to highlight CSV output (Alexandre

Rafalovitch)

* SOLR-9005: In files example, add a guard condition to javascript URP script

(Alexandre Rafalovitch)

* SOLR-9519: JSON Facet API: don't stop at an empty facet bucket if any sub-facets
still have a chance
of matching something due to filter exclusions (which can widen the domain
again).
(Michael Sun, yonik)

* SOLR-9740: A bug in macro expansion of multi-valued parameters caused non-

expanded values
after the first expanded value in the same multi-valued parameter to be dropped.
(Erik Hatcher, yonik)

* SOLR-9751: PreAnalyzedField can cause managed schema corruption. (Steve Rowe)

* SOLR-9736: Solr resolves the collection name against the first available leader
or first replica
of the first slice. This puts undue pressure on leader cores and likely on the
wrong ones. This is
fixed to randomly pick a leader on updates or a replica core otherwise. (Cao Manh
Dat via shalin)
* SOLR-9284: The HDFS BlockDirectoryCache should not let it's keysToRelease or
names maps grow indefinitely.
(Mark Miller, Michael Sun)

* SOLR-9729: JDBCStream improvements (Kevin Risden)

* SOLR-9626: new Admin UI now also highlights matched terms in the Analysis screen.
(Alexandre Rafalovitch)

* SOLR-9512: CloudSolrClient's cluster state cache can break direct updates to

leaders (noble)

* SOLR-5260: Facet search on a docvalue field in a multi shard collection (Trym

Møller, Erick Erickson)

* SOLR-9768: RecordingJsonParser produces incomplete json (Wojciech Stryszyk via

ab)

* SOLR-9616: Solr throws exception when expand=true on empty index (Timo Hund via
Ishan Chattopadhyaya)

* SOLR-9832: Schema modifications are not immediately visible on the coordinating

node. (Steve Rowe)

* SOLR-9834: A variety of spots in the code can create a collection zk node after
the collection has been
removed. (Mark Miller)

* SOLR-9707: Don't forward DeleteByQuery requests to down replicas. (Jessica Cheng

Mallet via Varun Thacker)

* SOLR-9823: CoreContainer incorrectly setting MDCLoggingContext for core (Jessica

Cheng Mallet via Erick Erickson)

* SOLR-1953: It may be possible for temporary files to accumulate until the Solr
process is shut down.
(Karl Wright, Mark Miller)

* SOLR-9847: Stop blocking further schema updates while waiting for a pending
update to propagate to other replicas.
This reduces the likelihood of a (time-limited) distributed deadlock during
concurrent schema updates.
(Mark Miller, Steve Rowe)

* SOLR-9760: Windows script doesn't need write permission (Alex Crome by Mikhail
Khludnev)

* SOLR-9699,SOLR-4668: fix exception from core status in parallel with core reload
(Mikhail Khludnev)

* SOLR-9859: replication.properties cannot be updated after being written and

neither replication.properties or
index.properties are durable in the face of a crash. (Pushkar Raste, Chris de
Kok, Cao Manh Dat, Mark Miller)

* SOLR-9901: Implement move in HdfsDirectoryFactory. (Mark Miller)

* SOLR-9900: fix false positives on range queries with

ReversedWildcardFilterFactory (Yonik Seeley via Mikhail Khludnev)
* SOLR-9495: AIOBE with confusing message for incomplete sort spec in Streaming
Expression (Gus Heck, Joel Bernstein)

* SOLR-9154: Fix DirectSolrSpellChecker to work when added through the Config API.
(Anshum Gupta)

* SOLR-9919: random Streaming Expression is not registered in /stream or /graph

handler (Joel Bernstein)

* SOLR-7495: Support Facet.field on a non-DocValued, single-value, int field (Varun

Thacker, Scott Stults)

* SOLR-9917: JSON Facet API percentile function caused a NullPointerException in

distributed mode when
there were no values in a bucket from a shard. (yonik)

* SOLR-9931: JSON Facet API hll (hyper-log-log) function returned 0 for non-empty
buckets with no field values
in local mode, but nothing for distributed mode. Both modes now return 0.
(yonik)

* SOLR-9503: NPE in Replica Placement Rules when using Overseer Role with other
rules (Tim Owen via noble)

* SOLR-9883: Example schemaless solr config files can lead to invalid tlog replays:
when updates are buffered,
update processors ordered before DistributedUpdateProcessor, e.g. field
normalization, are never run. (Steve Rowe)

* SOLR-9644: SimpleMLTQParser and CloudMLTQParser did not handle field boosts

properly
and CloudMLTQParser included extra strings from the field definitions in the
query.
(Ere Maijala via Anshum Gupta)

* SOLR-9954: Prevent against failure during failed snapshot cleanup from swallowing
the actual cause
for the snapshot to fail. (thelabdude)

Other Changes
----------------------

* SOLR-7539: Upgrade the clustering plugin to Carrot2 3.15.0. (Dawid Weiss)

* SOLR-9621: Remove several Guava & Apache Commons calls in favor of java 8
alternatives.
(Michael Braun via David Smiley)

* SOLR-9720: Refactor Responsewriters to remove dependencies on TupleStream,

Tuple, Explanation (noble)

* SOLR-9717: Refactor '/export' to not hardcode the JSON output and to use an API
(noble)

* SOLR-9739: JavabinCodec implements PushWriter interface (noble)

* SOLR-8332: Factor HttpShardHandler[Factory]'s url shuffling out into a

ReplicaListTransformer class.
 (Christine Poerschke, Noble Paul)

* SOLR-9597: Add setReadOnly(String ...) to ConnectionImpl (Kevin Risden)

* SOLR-9609: Change hard-coded keysize from 512 to 1024 (Jeremy Martini via Erick
Erickson)

* SOLR-8785: Use Dropwizard Metrics library for core metrics. The copied over code
in
org.apache.solr.util.stats has been removed. (Jeff Wartes, Kelvin Wong, Christine
Poerschke, shalin)

* SOLR-9784: Refactor CloudSolrClient to eliminate direct dependency on ZK (noble)

* SOLR-9801: Upgrade jetty to 9.3.14.v20161028 (shalin)

* SOLR-9783: (Search|Top)Group[s]ShardResponseProcessor.process: turned

sortWithinGroup null check into assert.
(Christine Poerschke)

* SOLR-9660: in GroupingSpecification factor [group](sort|offset|limit) into

[group](sortSpec)
(Judith Silverman, Christine Poerschke)

* SOLR-9819: Upgrade commons-fileupload to 1.3.2, fixing a potential vulnerability

CVE-2016-3092 (Anshum Gupta)

* SOLR-9827: ConcurrentUpdateSolrClient creates a RemoteSolrException if the remote

host responded with a non-ok
response (instead of a SolrException) and includes the remote error message as
part of the exception message
(Tomás Fernández Löbbe)

* SOLR-9846: OverseerAutoReplicaFailoverThread can take too long to stop and leak

out of unit tests. (Mark Miller)

* SOLR-8959: Refactored TestSegmentSorting out of TestMiniSolrCloudCluster

(hossman)

* SOLR-9874: Solr will reject CREATEALIAS requests if target collections don't

exist (Tomás Fernández Löbbe)

* SOLR-9878: fixing lazy logic for retrieving ReversedWildcardFilterFactory in

SolrQueryParserBase (Mikhail Khludnev)

* SOLR-9758: refactor preferLocalShards implementation (Christine Poerschke)

* SOLR-9448: providing a test to workaround a differently named uniqueKey field

(Mikhail Khludnev)

* SOLR-9899: StandardDirectoryFactory should use optimizations for all

FilterDirectorys not just NRTCachingDirectory.
(Mark Miller)

* SOLR-9915: PeerSync alreadyInSync check is not backwards compatible and results

in full replication during a rolling restart
(Tim Owen via noble)

* SOLR-3990: Moves getIndexSize() from ReplicationHandler to SolrCore (Shawn

Heisey)

* SOLR-9944: Map the nodes function name to the GatherNodesStream (Joel Bernstein)

* SOLR-9777: IndexFingerprinting should use getCombinedCoreAndDeletesKey() instead

of getCoreCacheKey() for per-segment caching (Ishan Chattopadhyaya)

* SOLR-9934: SolrTestCase.clearIndex has been improved to take advantage of low

level test specific logic that
clears the index metadata more completely then a normal *:* DBQ can due to update
versioning. (hossman)

* SOLR-9893: Update Mockito to version 2.6.2 for Java 9 support. Disable all legacy
EasyMock tests when running
on Java 9 until they were migrated to Mockito. (Uwe Schindler)

================== 6.3.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.12.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.8.v20160314

Detailed Change List

----------------------

Upgrade Notes
----------------------

* If you use the JSON Facet API (json.facet) with method=stream, you must now set
sort='index asc' to get the streaming
behavior; otherwise it won't stream. Reminder: "method" is a hint that doesn't
change defaults of other parameters.

* If you use the JSON Facet API (json.facet) to facet on a numeric field and if you
use mincount=0 or if you set the
prefix, then you will now get an error as these options are incompatible with
numeric faceting.

* Solr's logging verbosity at the INFO level has been greatly reduced, and
you may need to update the log configs to use the DEBUG level to see all the
logging messages you used to see at INFO level before.

* We are no longer backing up solr.log and solr_gc.log files in date-stamped copies

forever. If you relied on
the solr_log_<date> or solr_gc_log_<date> being in the logs folder that will no
longer be the case.
See SOLR-9570 for details.

* The create/deleteCollection methods on MiniSolrCloudCluster have been

deprecated. Clients should instead use the CollectionAdminRequest API. In
addition, MiniSolrCloudCluster#uploadConfigDir(File, String) has been
 deprecated in favour of #uploadConfigSet(Path, String)

* The bin/solr.in.sh (bin/solr.in.cmd on Windows) is now completely commented by

default. Previously, this wasn't so,
which had the effect of masking existing environment variables.

New Features
----------------------
* SOLR-5725: facet.method=enum can bypass exact counts calculation with
facet.exists=true, it just returns 1 for
terms which exists in result docset. (Alexey Kozhemiakin, Sebastian Koziel,
Radoslaw Zielinski via Mikhail Khludnev)

* SOLR-9127: Excel workbook (.xlsx) response writer. use 'wt=xlsx' (Tony Moriarty,
noble)

* SOLR-9469: JettySolrRunner now has the option of restarting using a different

port (Alan Woodward)

* SOLR-9319: DELETEREPLICA can accept a 'count' and remove appropriate replicas

(Nitin Sharma, noble)

* SOLR-8186: Reduce logging to logs/solr-<port>-console.log when not running in

foreground mode
Show timestamp also in foreground log. Also removes some logging noise. (janhoy)

* SOLR-8487: Adds CommitStream to support sending commits to a collection being

updated. (Dennis Gove)

* SOLR-9534: You can now set Solr's log level through environment variable
SOLR_LOG_LEVEL
Also adds conveience arguments -q (quiet: WARN) and -v (verbose: DEBUG) to
bin/solr (janhoy)

* SOLR-9537: Support facet scoring with the scoreNodes expression (Joel Bernstein)

* SOLR-9558: DIH TemplateTransformerto to support multivalued fields (Ted Sullivan

via noble)

* SOLR-9557: Every implicit requesthandler now has a default 'useParams' attribute

(noble)

* SOLR-9572: config API to show expanded useParams for request handlers inline
(noble)

* SOLR-9258: Optimizing, storing and deploying AI models with Streaming Expressions

(Cao Manh Dat, Joel Bernstein)

* SOLR-9205: Added method LukeResponse.getSchemaFlags() which returns field

information as an EnumSet (Fengtan, Alan Woodward)

* SOLR-9520: Kerberos delegation support in SolrJ (Ishan Chattopadhyaya, noble)

* SOLR-9146: Parallel SQL engine should support >, >=, <, <=, <>, != syntax
(Timothy Potter, Joel Bernstein, Kevin Risden)

* SOLR-9337: Add fetch Streaming Expression (Joel Bernstein)

* SOLR-9103: Restore ability for users to add custom Streaming Expressions (Cao
Manh Dat)

* SOLR-9657: New TemplateUpdateProcessorFactory added (noble)

* SOLR-9417: Allow daemons to terminate when they finish iterating a topic (Joel
Bernstein)

* SOLR-8370: Display configured Similarity in Schema-Browser, both global/default

and per-field/field-type
(janhoy, Alexandre Rafalovitch)

* SOLR-9326: Ability to create/delete/list snapshots at collection level.

(Hrishikesh Gadre via yonik)

* SOLR-9662: New parameter -u <user:pass> in bin/post to pass basicauth credentials

(janhoy)

* SOLR-9654: Add "overrequest" parameter to JSON Facet API to control amount of

overrequest
on a distributed terms facet. (yonik)

* SOLR-2212: Add a factory class corresponding to Lucene's NoMergePolicy. (Lance

Norskog, Cao Manh Dat via shalin)

* SOLR-9670: Support SOLR_AUTHENTICATION_OPTS in solr.cmd (janhoy)

* SOLR-9559: Add ExecutorStream to execute stored Streaming Expressions (Joel

Bernstein)

* SOLR-1085: Add support for MoreLikeThis queries and responses in SolrJ client.
(Maurice Jumelet, Bill Mitchell, Cao Manh Dat via shalin)

Bug Fixes
----------------------

* SOLR-9310: PeerSync fails on a node restart due to IndexFingerPrint mismatch

(Pushkar Raste, noble)

* SOLR-9484: The modify collection API should wait for the modified properties to
show up in the
cluster state. (Cao Manh Dat, shalin)

* SOLR-9507: CoreContainer threads now correctly set their MDC logging values
(Alan Woodward)

* SOLR-9522: Improve error handling in ZKPropertiesWriter (Varun Thacker)

* SOLR-8080: bin/solr start script now exits with informative message if using
wrong Java version (janhoy)

* SOLR-9475: bin/install_solr_service.sh script got improved detection of Linux

distro, especially within
virtualized/Docker environment through parsing of /etc/*-release files. Now also
supports CentOS. (janhoy)

* SOLR-9524: SolrIndexSearcher.getIndexFingerprint uses dubious synchronization

(Mike Drob, noble)

* SOLR-9542: Kerberos delegation tokens requires Jackson library (Ishan

Chattopadhyaya via noble)

* SOLR-9330: Fix AlreadyClosedException on admin/mbeans?stats=true (Mikhail

Khludnev)

* SOLR-9411: Better validation for Schema API add-field and add-dynamic-field

(janhoy, Steve Rowe)

* SOLR-9504: A replica with an empty index becomes the leader even when other more
qualified replicas
are in line. (shalin)

* SOLR-9554: Fix NullPointerException when cores are loaded in parallel and switch
schema.xml to managed-scheme.
(Alan Woodward, Mikhail Khludnev)

* SOLR-9556: OverseerAutoFailoverReplicaThread was not exiting on interrupt

(Alan Woodward)

* SOLR-9563: Collection creation could fail if an ADDREPLICA subrequest arrived

at a node before its local state had updated with the new collection data
(Alan Woodward)

* SOLR-9278: Index replication interactions with IndexWriter can cause deadlock.

(Xunlong via Mark Miller)

* SOLR-9604: Pooled SSL connections were not being re-used (Alan Woodward,
Mikhail Khludnev, hossman)

* SOLR-9325: solr.log is now written to $SOLR_LOGS_DIR without changing

log4j.properties (janhoy)

* SOLR-9518: Kerberos Delegation Tokens don't work without a chrooted ZK (Ishan

Chattopadhyaya,via noble)

* SOLR-9687: Fixed Interval Facet count issue in cases of open/close intervals on

the same values
(Andy Chillrud, Tomás Fernández Löbbe)

* SOLR-9441: Solr collection backup on HDFS can only be manipulated by the Solr
process owner.
(Hrishikesh Gadre via Mark Miller)

* SOLR-9536: OldBackupDirectory timestamp field needs to be initialized to avoid

NPE.
(Hrishikesh Gadre, hossman via Mark Miller)

* SOLR-2039: Multivalued fields with dynamic names does not work properly with DIH.
(K A, ruslan.shv, Cao Manh Dat via shalin)

* SOLR-4164: group.limit=-1 was not supported for grouping in distributed mode.

(Cao Manh Dat, Lance Norskog, Webster Homer, hossman, yonik)

* SOLR-9692: blockUnknown property makes inter-node communication impossible

(noble)

* SOLR-2094: XPathEntityProcessor should reinitialize the XPathRecordReader

instance if
the 'forEach' or 'xpath' attributes are templates & it is not a root entity (Cao
Manh Dat, noble)

* SOLR-9697: zk upconfig broken on windows (Xavier Jmlucjav via janhoy)

Optimizations
----------------------

* SOLR-9374: Speed up Jmx MBean retrieval for FieldCache. (Tim Owen via shalin)

* SOLR-9449: Example schemas do not index _version_ field anymore because the field
has DocValues enabled already. (shalin)

* SOLR-9447: Do not clone SolrInputDocument if update processor chain does not

contain custom processors.
(shalin)

* SOLR-9452: JsonRecordReader should not deep copy document before

handler.handle(). (noble, shalin)

* SOLR-9142: JSON Facet API: new method=dvhash can be chosen for fields with high
cardinality. (David Smiley)

* SOLR-9446: Leader failure after creating a freshly replicated index can send
nodes into recovery even if
index was not changed (Pushkar Raste, noble)

* SOLR-9592: retrieving docValues as stored values was sped up by using the proper
leaf
reader rather than ask for a global view. In extreme cases, this leads to a 100x
speedup.
(Takahiro Ishikawa, yonik)

* SOLR-9566: Don't put replicas into recovery when first creating a Collection
(Alan Woodward)

* SOLR-9546: Eliminate unnecessary boxing/unboxing going on in SolrParams (Pushkar

Raste, noble)

* SOLR-9506: cache IndexFingerprint for each segment (Pushkar Raste, yonik, noble)

* SOLR-7506: Roll over GC logs by default via bin/solr scripts (shalin, janhoy)

Other Changes
----------------------

* SOLR-9412: Add failOnMissingParams option to MacroExpander, add TestMacroExpander

class.
(Jon Dorando, Christine Poerschke)

* SOLR-9406: SolrSuggester should selectively register close hook (Gethin James,

Joel Bernstein)

* SOLR-8961: Add a test module for solr-test-framework (Alan Woodward)

* SOLR-9474: MiniSolrCloudCluster will not reuse ports by default when

restarting its JettySolrRunners (Alan Woodward)

* SOLR-9498: Remove HDFS properties from DIH solrconfig.xml, as started in SOLR-

6943 (Alexandre Rafalovitch)
* SOLR-9365: Reduce noise in solr logs during graceful shutdown. (Cao Manh Dat via
shalin)

* SOLR-9451: Make clusterstatus command logging less verbose. (Varun Thacker)

* SOLR-9502: ResponseWriters should natively support MapSerializable (noble)

* SOLR-9538: Relocate (BinaryResponse|JSON|Smile)Writer tests to

org.apache.solr.response
which is the package of the classes they test. (Jonny Marks via Christine
Poerschke)

* SOLR-9508: Install script install_solr_service.sh now checks existence of tools.

New option -n to avoid starting service after installation (janhoy)

* SOLR-7826: Refuse "bin/solr create" if run as root, unless -force is specified

(janhoy, Binoy Dalal)

* SOLR-6871: Updated the quickstart tutorial to cover the 6.2.0 release, and added
ant target
"generate-website-quickstart" to convert the bundled version of the tutorial into
one suitable
for the website.

* SOLR-5563: Move lots of SolrCloud logging from 'info' to 'debug' (janhoy, Alan
Woodward)

* SOLR-9544: Allow ObjectReleaseTracker more time to check for asynchronously

closing resources (Alan Woodward)

* SOLR-6677: Reduced logging during startup and shutdown, moved more logs to DEBUG
level
(janhoy, Shawn Heisey, Alan Woodward)

* SOLR-6090: Remove unreachable printLayout usage in cloud tests. (Cao Manh Dat via
shalin)

* SOLR-9551: Add JSONWriter constructor variant,

JSONWriterTest.testConstantsUnchanged test.
(Jonny Marks, Christine Poerschke)

* SOLR-9500: Add a LogLevel annotation to set log levels on specific tests (Alan
Woodward)

* SOLR-9548: The beginning of solr.log now starts with a more informative welcome
message (janhoy)

* SOLR-9547: Do not allow bin/solr start as root user, unless -force param
specified (janhoy)

* SOLR-9567: Make ReRankQParserPlugin's private ReRankCollector a public class of

its own. (Christine Poerschke)

* SOLR-7436: Solr stops printing stacktraces in log and output (janhoy, hossman,
Markus Jelsma)

* SOLR-9576: Make FieldAnalysisRequestHandler, DocumentAnalysisRequestHandler &

DumpRequestHandler
 implicit (noble)

* SOLR-9574: Factor out AbstractReRankQuery from ReRankQParserPlugin's private

ReRankQuery. (Christine Poerschke)

* SOLR-5041: Add a test to make sure that a leader always recovers from log on
startup. (Cao Manh Dat, shalin)

* SOLR-9588: Remove Guava dependency from SolrJ (Ishan Chattopadhyaya, noble)

* SOLR-8140: Remove mentions of unimplemented admin-extra from the new Admin UI

(Alexandre Rafalovitch)

* SOLR-9589: Remove jackson dependency from SolrJ (Ishan Chattopadhyaya, noble)

* SOLR-8385: Narrow StreamFactory.withFunctionName clazz parameter to prevent

misconfiguration (Jason Gerlowski, Kevin Risden)

* SOLR-8969: SQLHandler causes NPE in non-cloud mode (Markus Jelsma, Kevin Risden)

* SOLR-9610: New AssertTool in SolrCLI for easier cross platform assertions from
command line (janhoy)

* SOLR-9680: Better error messages in SolrCLI when authentication required (janhoy)

* SOLR-9639: Test only fix. Prevent CDCR tests from removing collection during
recovery that used to blow up jvm (Mikhail Khludnev)

* SOLR-9625: Add HelloWorldSolrCloudTestCase class (Christine Poerschke, Alan

Woodward, Alexandre Rafalovitch)

* SOLR-9642: Refactor the core level snapshot cleanup mechanism to rely on Lucene
(Hrishikesh Gadre via yonik)

* SOLR-9627: Add QParser.getSortSpec, deprecate misleadingly named QParser.getSort

(Judith Silverman, Christine Poerschke)

* SOLR-9632: Add MiniSolrCloudCluster#deleteAllCollections() method (Alan Woodward)

* SOLR-9634: Deprecate collection methods on MiniSolrCloudCluster (Alan Woodward)

* SOLR-7850: Moved defaults within bin/solr.in.sh (and bin/solr.in.cmd on Windows)

to bin/solr (and bin/solr.cmd)
such that the default state of these files is to set nothing. This makes Solr
work better with Docker. (David Smiley)

* SOLR-9570: Various log tidying now happens at Solr startup:

Old solr_log_<date> and solr_gc_log_<date> files are removed, avoiding disks to
fill up,
solr.log.X files are rotated, preserving solr.log from last run in solr.log.1,
solr.log.1 => solr.log.2 etc
solr-*-console.log files are moved into $SOLR_LOGS_DIR/archived/ instead of being
overwritten
Last JVM garbage collection log solr_gc.log is moved into
$SOLR_LOGS_DIR/archived/
(janhoy)

* SOLR-4531: Add tests to ensure that recovery does not fail on corrupted tlogs.
(Simon Scofield, Cao Manh Dat via shalin)
* SOLR-5245: Add a test to ensure that election contexts are keyed off both
collection name and coreNodeName
so that killing a shard in one collection does not result in leader election in a
different collection.
See SOLR-5243 for the related bug. (Cao Manh Dat via shalin)

* SOLR-9533: Reload core config when a core is reloaded (Gethin James, Joel
Bernstein)

* SOLR-9371: Fix bin/solr calculations for start/stop wait time and RMI_PORT.
(Shawn Heisey via Erick Erickson)

================== 6.2.1 ==================

Bug Fixes
----------------------

* SOLR-9494: Use of {!collapse} sometimes doesn't correctly return true for

Collector.needsScores(), especially when the
query was cached. This can cause an exception when 'q' is a SpanQuery or
potentially others. (David Smiley)

* SOLR-6744: fl renaming / alias of uniqueKey field generates null pointer

exception in SolrCloud configuration
(Mike Drob via Tomás Fernández Löbbe)

* SOLR-9445: Admin requests are retried by CloudSolrClient and LBHttpSolrClient on

failure. (shalin)

* SOLR-9439: Shard split clean up logic for older failed splits is faulty. The
delete shard API
has also been made more resilient against failures resulting from non-existent
cores. (shalin)

* SOLR-9430: Fix locale lookup in DIH <propertyWriter/> to use BCP47 language tags
to be consistent with other places in Solr. Language names still work for
backwards
compatibility. (Uwe Schindler, Boris Steiner)

* SOLR-9389: HDFS Transaction logs stay open for writes which leaks Xceivers. (Tim
Owen via Mark Miller)

* SOLR-9188: blockUnknown property makes inter-node communication impossible

(noble)

* SOLR-9455: Deleting a sub-shard in recovery state can mark parent shard as

inactive. (shalin)

* SOLR-9461: DELETENODE, REPLACENODE should pass down the 'async' param to

subcommands (shalin, noble)

* SOLR-9444: Fix path usage for cloud backup/restore. (Hrishikesh Gadre, Uwe
Schindler, Varun Thacker)

* SOLR-9381: Snitch for freedisk uses '/' instead of 'coreRootDirectory' (Tim Owen,
noble)

* SOLR-9488: Shard split can fail to write commit data on shutdown/restart causing
replicas to recover
without replicating the index. This can cause data loss. (shalin)

* SOLR-9490: Fixed bugs in BoolField that caused it to erroneously return "false"

for all docs depending
on usage (Colvin Cowie, Dan Fox, hossman)

* SOLR-9438: Shard split can be marked successful and sub-shard states switched to
'active' even when
one or more sub-shards replicas do not recover due to the leader crashing or
restarting between the time
the replicas are created and before they can recover. This can cause data loss.
(shalin)

* SOLR-9408: Fix TreeMergeOutputFormat to add timestamp metadata to a commit.

SolrCloud replication relies on this.
(Jessica Cheng Mallet via Varun Thacker)

Other Changes
----------------------

* SOLR-7362: Fix TestReqParamsAPI test failures (noble, Varun Thacker)

================== 6.2.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.12.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.8.v20160314

Detailed Change List

----------------------

New Features
----------------------

* SOLR-9187: Support dates and booleans in /export handler, support boolean

DocValues fields (Erick Erickson)

* SOLR-8048: bin/solr script should support basic auth credentials provided in

solr.in.sh (noble)

* SOLR-7374: Core level Backup/Restore now supports specifying the directory

implementation to use
via the "repository" parameter. (Hrishikesh Gadre, Varun Thacker, Mark Miller)

* SOLR-9216: Support collection.configName in MODIFYCOLLECTION request (Keith

Laban, noble)

* SOLR-9251: Support for a new tag 'role' in replica placement rules (noble)

* SOLR-9194: Enhance the bin/solr script to perform file operations to/from

Zookeeper (Erick Erickson, janhoy)

* SOLR-9242: Collection Backup/Restore now supports specifying the directory

implementation to use
via the "repository" parameter. (Hrishikesh Gadre, Varun Thacker)

* SOLR-9193: Add scoreNodes Streaming Expression (Joel Bernstein)

* SOLR-9243: Add terms.list parameter to the TermsComponent to fetch the docFreq

for a list of terms
(Joel Bernstein)

* SOLR-9090: Add directUpdatesToLeadersOnly flag to solrj CloudSolrClient.

(Marvin Justice, Christine Poerschke)

* SOLR-9270: Allow spatialContextFactory to be simply "JTS". And if any spatial

params include the old
Spatial4j package "com.spatial4j.core" it is rewritten to
"org.locationtech.spatial4j" with a warning.
(David Smiley)

* SOLR-9240: Support parallel ETL with the topic expression (Joel Bernstein)

* SOLR-9275: XML QueryParser support (defType=xmlparser) now extensible via

configuration.
(Christine Poerschke)

* SOLR-9200: Add Delegation Token Support to Solr.

(Gregory Chanan)

* SOLR-9038: Solr core snapshots: The current commit can be snapshotted which
retains the commit and associates it with
a name. The core admin API can create snapshots, list them, and delete them.
Snapshot names can be referenced in
doing a core backup, and in replication. Snapshot metadata is stored in a new
snapshot_metadata/ dir.
(Hrishikesh Gadre via David Smiley)

* SOLR-9279: New boolean comparison function queries comparing numeric arguments:

gt, gte, lt, lte, eq
(Doug Turnbull, David Smiley)

* SOLR-9252: Feature selection and logistic regression on text (Cao Manh Dat, Joel
Bernstein)

* SOLR-6465: CDCR: fall back to whole-index replication when tlogs are

insufficient.
(Noble Paul, Renaud Delbru, shalin)

* SOLR-9320: A REPLACENODE command to decommission an existing node with another

new node
(noble, Nitin Sharma, Varun Thacker)

* SOLR-9318: A DELETENODE command to delete all replicas in that node (noble, Nitin
Sharma, Varun Thacker)

Bug Fixes
----------------------

* SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed (Scott Blum, Noble Paul)

* SOLR-9199: ZkController#publishAndWaitForDownStates logic is inefficient

(Hrishikesh Gadre)

* SOLR-9161: Change SolrPluginUtils.invokeSetters implementation to accommodate

setter variants.
(Christine Poerschke, Steve Rowe, Uwe Schindler)

* SOLR-9234: srcField parameter works only when all fields are captured in the
/update/json/docs
endpoint (noble)

* SOLR-8546: SOLR-8546: TestLazyCores is failing a lot on the Jenkins cluster.

(Erick Erickson)

* SOLR-9237: DefaultSolrHighlighter.doHighlightingByFastVectorHighlighter can't be

overidden (janhoy)

* SOLR-8626: 404 error when clicking nodes in cloud graph view in angular UI.
(janhoy, Trey Grainger via shalin)

* SOLR-9254: GraphTermsQueryQParserPlugin throws NPE when field being searched is

not present in segment
(Joel Bernstein)

* SOLR-8657: Fix SolrRequestInfo error logs if QuerySenderListener is being used

(Pascal Chollet,
Tomás Fernández Löbbe)

* SOLR-8777: Duplicate Solr process can cripple a running process. (Jessica Cheng
Mallet, Scott Blum, shalin)

* SOLR-9246: If the JDBCStream sees an unknown column type it will now throw a
detailed exception. (Dennis Gove)

* SOLR-9181: Fix some races in CollectionStateWatcher API (Alan Woodward, Scott

Blum)

* SOLR-9235: Fixed NPE when using non-numeric range query in deleteByQuery

(hossman)

* SOLR-9088: Fixed TestManagedSchemaAPI failures which exposed race conditions in

the schema API ( Varun Thacker, noble)

* SOLR-9207: PeerSync recovery failes if number of updates requested is high. A new

useRangeVersions config option
is introduced (defaults to true) to send version ranges instead of individual
versions for peer sync.
(Pushkar Raste, shalin)

* SOLR-8858: SolrIndexSearcher#doc() completely ignores field filters unless lazy

field loading is enabled.
(Caleb Rackliffe, David Smiley, shalin)

* SOLR-9236: AutoAddReplicas will append an extra /tlog to the update log location
on replica failover.
(Eungsop Yoo, Mark Miller)

* SOLR-9291: ZkSolrResourceLoader should not retry fetching resources if the server

has been shutdown.
(shalin)
* SOLR-9287: Including 'score' in the 'fl' param when doing an RTG no longer causes
an NPE
(hossman, Ishan Chattopadhyaya)

* SOLR-7280: In cloud-mode sort the cores smartly before loading & limit threads to
improve cluster stability
(noble, Erick Erickson, shalin)

* SOLR-9285: Fixed AIOOBE when using ValueSourceAugmenter in single node RTG

(hossman)

* SOLR-9288: Fix [docid] transformer to return -1 when used in RTG with uncommitted
doc (hossman)

* SOLR-9309: Fix SolrCloud RTG response structure when multi ids requested but only
1 found (hossman)

* SOLR-9334: CloudSolrClient.collectionStateCache is unbounded (noble)

* SOLR-9339: NPE in CloudSolrClient when the response is null (noble)

* SOLR-8596: Web UI doesn't correctly generate queries which include local

parameters (Alexandre Rafalovitch, janhoy)

* SOLR-8645: managed-schema is now syntax highlighted in cloud->Tree view

(Alexandre Rafalovitch via janhoy)

* SOLR-8379: UI Cloud->Tree view now shows .txt files correctly (Alexandre

Rafalovitch via janhoy)

* SOLR-9003: New Admin UI's Dataimport screen now correctly displays DIH Debug
output (Alexandre Rafalovitch)

* SOLR-9308: Fix distributed RTG to forward request params, fixes fq and non-
default fl params (hossman)

* SOLR-9179: NPE in IndexSchema using IBM JDK (noble, Colvin Cowie)

* SOLR-9397: Config API does not support adding caches (noble)

* SOLR-9405: ConcurrentModificationException in ZkStateReader.getStateWatchers.

(Alan Woodward, Edward Ribeiro, shalin)

* SOLR-9232: Admin UI now fully implements Swap Cores interface (Alexandre

Rafalovitch)

* SOLR-8715: Admin UI's Schema screen now works for fields with stored=false and
some content indexed (Alexandre Rafalovitch)

* SOLR-8911: In Admin UI, enable scrolling for overflowing Versions and JVM
property values (Alexandre Rafalovitch)

* SOLR-9002: Admin UI now correctly displays json and text files in the
collection/Files screen (Upayavira, Alexandre Rafalovitch)

* SOLR-8993: Admin UI now correctly supports multiple DIH handler end-points

(Upayavira, Alexandre Rafalovitch)
* SOLR-9032: Admin UI now correctly implements Create Alias command (Upayavira,
Alexandre Rafalovitch)

* SOLR-9391: LBHttpSolrClient.request now correctly returns Rsp.server when

previously skipped servers were successfully tried. (Christine Poerschke)

Optimizations
----------------------

* SOLR-9219: Make hdfs blockcache read buffer sizes configurable and improve cache
concurrency. (Mark Miller)

* SOLR-9264: Optimize ZkController.publishAndWaitForDownStates to not read all

collection states and
watch relevant collections instead. (Hrishikesh Gadre, shalin)

* SOLR-9335: Solr cache/search/update stats counters now use LongAdder which are
supposed to have higher throughput
under high contention. (Varun Thacker)

* SOLR-9350: JSON Facets: method="stream" will no longer always uses & populates
the filter cache, likely
flushing it. 'cacheDf' can be configured to set a doc frequency threshold, now
defaulting to 1/16th doc count.
Using -1 Disables use of the cache. (David Smiley, yonik)

Other Changes
----------------------

* SOLR-9195: Remove unnecessary allocation and null check in

UpdateRequestProcessorChain's
getReqProcessors method. (Christine Poerschke)

* SOLR-8981: Upgraded Extraction module to Apache Tika 1.13.

(Tim Allison, Lewis John McGibbney via Uwe Schindler)

* SOLR-9076: Update to Hadoop 2.7.2

(Mark Miller, Gregory Chanan)

* SOLR-8787: TestAuthenticationFramework should not extend

TestMiniSolrCloudCluster. (Trey Cahill via shalin)

* SOLR-9180: More comprehensive tests of pseudo-fields for RTG and SolrCloud

requests (hossman)

* SOLR-7930: Comment out trappy references to example docs in elevate.xml files

(Erick Erickson)

* SOLR-9277: Clean up some more remnants of supporting old and new style solr.xml
in tests (Erick Erickson)

* SOLR-9163: Sync up basic_configs and data_driven_schema_configs, removing almost

all differences
except what is required for schemaless. (yonik)

* SOLR-9340: Change ZooKeeper disconnect and session expiry related logging from
INFO to WARN to
make debugging easier (Varun Thacker)
* SOLR-9358: [AngularUI] In Cloud->Tree file view area, collapse metadata by
default (janhoy)

* SOLR-9256: asserting hasNext() contract in JdbcDataSource in DataImportHandler

(Kristine Jetzke via Mikhai Khludnev)

* SOLR-9209: extracting JdbcDataSource.createResultSetIterator() for extension

(Kristine Jetzke via Mikhai Khludnev)

* SOLR-9353: Factor out ReRankQParserPlugin.ReRankQueryRescorer private class.

(Christine Poerschke)

* SOLR-9392: Fixed CDCR Test failures which were due to leaked resources. (shalin)

* SOLR-9385: Add QParser.getParser(String,SolrQueryRequest) variant. (Christine

Poerschke)

* SOLR-9367: Improved TestInjection's randomization logic to use

LuceneTestCase.random() (hossman)

* SOLR-9331: Remove ReRankQuery's length constructor argument and member.

(Christine Poerschke)

* SOLR-9092: For the delete replica command we attempt to send the core admin
delete request only
if that node is actually up. (Jessica Cheng Mallet, Varun Thacker)

* SOLR-9410: Make ReRankQParserPlugin's private ReRankWeight a public class of its

own. (Christine Poerschke)

* SOLR-9404: Refactor move/renames in JSON FacetProcessor and FacetFieldProcessor.

(David Smiley)

* SOLR-9421: Refactored out OverseerCollectionMessageHandler to smaller classes

(noble)

* SOLR-8643: BlockJoinFacetComponent is substituted by

BlockJoinFacetDocSetComponent. It doesn't need to change solrconfig.xml (Mikhail
Khludnev)

* SOLR-8644: Test asserts that block join facets work with parent level fq
exclusions. (Dr. Oleg Savrasov via Mikhail Khludnev)

================== 6.1.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.12.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.8.v20160314

Upgrading from Solr any prior release

----------------------

* If you use historical dates, specifically on or before the year 1582, you should
re-index.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-8782: Add asynchronous sugar methods to the SolrJ Collections API. You
can now call .processAsync() to run a method asynchronously, or
.processAndWait() to wait for a call to finish without holding HTTP
collections open. (Alan Woodward)

* SOLR-8765: Enforce required parameters at query construction time in the SolrJ

Collections API, add static factory methods, and deprecate old setter methods.
(Alan Woodward, Jason Gerlowski)

* SOLR-8842: authorization APIs do not use name as an identifier for a permission

for update, delete commands and 'before' attribute (noble)

* SOLR-8814: Support GeoJSON response writer and general spatial formatting.

Adding
&wt=geojson&geojson.field=<your geometry field>
Will return a FeatureCollection for each SolrDocumentList and a Feature with the
requested geometry for each SolrDocument. The requested geometry field needs
to either extend AbstractSpatialFieldType or store a GeoJSON string. This also
adds
a [geo] DocumentTransformer that can return the Shape in a variety of formats:
&fl=[geo f=<your geometry field> w=(GeoJSON|WKT|POLY)]
The default format is GeoJSON. For information on the supported formats, see:
https://github.com/locationtech/spatial4j/blob/master/FORMATS.md
To return the FeatureCollection as the root element, add '&omitHeader=true"
(ryan)

* SOLR-8859: Spatial fields like RPT can now be configured to use Spatial4j
registered shape formats
e.g. via format="GeoJSON". (ryan, David Smiley)

* SOLR-445: new TolerantUpdateProcessorFactory to support skipping update commands

that cause
failures when sending multiple updates in a single request.
(Erick Erickson, Tomás Fernández Löbbe, Anshum Gupta, hossman)

* SOLR-8890: New static method in DistributedUpdateProcessorFactory to allow

UpdateProcessorFactories
to indicate request params that should be forwarded when DUP distributes updates.
(hossman)

* SOLR-8888: Add shortestPath Streaming Expression. This expression performs a

breadth first
graph traversal to find the shortest path(s) in an unweighted, directed graph
stored in a
SolrCloud collection. (Joel Bernstein)

* SOLR-8938: Add optional --excluderegex argument to ZkCLI. (Christine Poerschke)

* SOLR-8976: Add SolrJ support for REBALANCELEADERS Collections API (Anshum Gupta)

* SOLR-8962: Add sort Streaming Expression. The expression takes a single input
stream and a
comparator and outputs tuples in stable order of the comparator. (Dennis Gove)

* SOLR-8349: Allow sharing of large in memory data structures across cores (Gus
Heck, noble)

* SOLR-9009: Adds ability to get an Explanation of a Streaming Expression (Dennis

Gove)

* SOLR-8918: Adds Streaming to the admin page under the collections section.
Includes
ability to see graphically the expression explanation (Dennis Gove)

* SOLR-8913: When using a shared filesystem we should store data dir and tlog dir
locations in
the cluster state. (Mark Miller)

* SOLR-8809: Implement Connection.prepareStatement (Kevin Risden)

* SOLR-9020: Implement StatementImpl/ResultSetImpl get/set fetch* methods and

proper errors for traversal methods (Kevin Risden)

* SOLR-9041: 'core-admin-read' and 'core-admin-edit' are well known permissions

(noble)

* SOLR-8986: Add Random Streaming Expression (Joel Bernstein)

* SOLR-8925: Add gatherNodes Streaming Expression to support breadth first

traversals (Joel Bernstein)

* SOLR-9027: Add GraphTermsQuery to limit traversal on high frequency nodes (Joel

Bernstein, David Smiley)

* SOLR-5750: Add /admin/collections?action=BACKUP and RESTORE assuming access to a

shared file system.
(Varun Thacker, David Smiley)

* SOLR-9049: RuleBasedAuthorizationPlugin supports regex in param values eg:

"command" : "REGEX:(i?)create" (noble)

* SOLR-8972: Add GraphHandler and GraphMLResponseWriter to support graph

visualizations (Joel Bernstein)

* SOLR-9026: Extend facet telemetry support to legacy (non-json) facets under

"debug/facet-debug" in
the response. (Michael Sun, yonik)

* SOLR-7117: Provide an option to limit the maximum number of cores that can be
created on a node by the
Auto Add Replica feature. For this you can set a "maxCoresPerNode" property via
the Cluster Property API
(Varun Thacker, Mark Miller)

* SOLR-8208: [subquery] document transformer executes separate requests per result

document. (Cao Manh Dat via Mikhail Khludnev)
* SOLR-8323, SOLR-9113: Add CollectionStateWatcher API (Alan Woodward, Scott Blum)

* SOLR-8988: Adds query option facet.distrib.mco which when set to true allows the
use of facet.mincount=1 in cloud mode.
(Keith Laban, Dennis Gove)

* SOLR-8583: Apply highlighting to hl.alternateField by default for Default and

FastVectorHighlighter.
Turn off with hl.highlightAlternate=false (janhoy, David Smiley)

* SOLR-7123: '/update/json/docs' path supports nested documents (noble)

* SOLR-8610: Resolve variables in encryptKeyFile of DIH's JdbcDataSource (Kristine

Jetzke via Mikhail Khludnev)

* SOLR-7739: Add a new ClassificationUpdateProcessorFactory. (Alessandro Benedetti

via Tommaso Teofili)

Bug Fixes
----------------------

* SOLR-8855: The HDFS BlockDirectory should not clean up it's cache on shutdown.
(Mark Miller)

* SOLR-8948: OverseerTaskQueue.containsTaskWithRequestId encounters json parse

error if a
SolrResponse node is in the overseer queue. (Jessica Cheng Mallet via shalin)

* SOLR-7729: ConcurrentUpdateSolrClient ignores the collection parameter in some

methods.
(Nicolas Gavalda, Jorge Luis Betancourt Gonzalez via Mark Miller)

* SOLR-8662: SchemaManager waits correctly for replicas to be notified of a new

change.
(sarowe, Noble Paul, Varun Thacker)

* SOLR-8983: Cleanup clusterstate and replicas for a failed create collection

request
(Varun Thacker, Anshum Gupta)

* SOLR-9029: fix rare ZkStateReader visibility race during collection state format
update (Scott Blum, hossman)

* SOLR-9046: Fix solr.cmd that wrongly assumes Jetty will always listen on 0.0.0.0.
(Bram Van Dam, Uwe Schindler)

* SOLR-9064: Adds an explanation of the incoming stream to an UpdateStream's

explanation (Dennis Gove)

* SOLR-9128: Fix error handling issues in Streaming classes (Joel Bernstein)

* SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD
(janhoy)

* SOLR-9141: Fix ClassCastException when using the /sql handler count() function
with
single-shard collections (Minoru Osuka via James Dyer)

* SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used

with "cursorMark".
(James Dyer)

* SOLR-8940: Fix group.sort option (hossman)

* SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH)

(Kristine Jetzke via Mikhail Khludnev)

* SOLR-8676: keep LOG4J_CONFIG in solr.cmd (Kristine Jetzke via Mikhail Khludnev)

* SOLR-9198: config APIs unable to add multiple values with same name (noble)

* SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided

and there are explicit operators (except for AND) - addresses problems caused by
SOLR-2649.
(Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe)

* SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to

FCS, causing slowdowns (Alessandro Benedetti, Jesse McLaughlin, Alan Woodward)

Optimizations
----------------------
* SOLR-8722: Don't force a full ZkStateReader refresh on every Overseer operation.
(Scott Blum via shalin)

* SOLR-8745: Deprecate costly ZkStateReader.updateClusterState(), replace with a

narrow
forceUpdateCollection(collection) (Scott Blum via shalin)

* SOLR-8856: Do not cache merge or 'read once' contexts in the hdfs block cache.
(Mark Miller, Mike Drob)

* SOLR-8922: Optimize filter creation (DocSetCollector) to minimize the amount of

garbage
produced. This resulted in up to 3x throughput when small filter creation was the
bottleneck,
as well as orders of magnitude less garbage. (Jeff Wartes, yonik)

* SOLR-8937: bin/post (SimplePostTool) now streams the standard input instead of

buffering fully.
(David Smiley)

* SOLR-8973: Zookeeper frenzy when a core is first created. (Janmejay Singh, Scott
Blum, shalin)

* SOLR-9014: Deprecate and reduce usage of ClusterState methods which may make
calls to ZK via
the lazy collection reference. (Scott Blum, shalin)

* SOLR-9106: Cluster properties are now cached on ZkStateReader. (Alan Woodward)

* SOLR-9147: Upgrade commons-io to 2.5, avoid expensive array resizing in

EmbeddedSolrServer (Mikhail Khludnev)

* SOLR-8744: Overseer operations performed with fine grained mutual exclusion

(noble, Scott Blum)

* SOLR-9204: Improve performance of getting directory size with hdfs. (Mark Miller)
Other Changes
----------------------
* SOLR-8860: Remove back-compat handling of router format made in SOLR-4221 in
4.5.0. (shalin)

* SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to

serialize a value.
(David Smiley)

* SOLR-8842: security rules made more foolproof by asking the requesthandler about
the well known
permission name. The APIs are also modified to ue 'index' as the unique
identifier instead of name.
Name is an optional attribute now and only to be used when specifying well-known
permissions (noble)

* SOLR-5616: Simplifies grouping code to use ResponseBuilder.needDocList() to

determine if it needs to
generate a doc list for grouped results. (Steven Bower, Keith Laban, Dennis Gove)

* SOLR-8869: Optionally disable printing field cache entries in SolrFieldCacheMBean

(Gregory Chanan)

* SOLR-8892: Allow SolrInfoMBeans to return different statistics for /jmx vs web ui

calls.
(Gregory Chanan, Mark Miller)

* SOLR-8097: Implement builder pattern design for constructing SolrJ clients and
also deprecate direct construction
of client objects. (Jason Gerlowski, Shawn Heisey, Anshum Gupta)

* SOLR-9015: Adds SelectStream as a default function in the StreamHandler (Dennis

Gove)

* SOLR-8929: Add an idea module for solr/server to enable launching start.jar

(Scott Blum, Steve Rowe)

* SOLR-8933: Solr should not close container streams. (Mike Drob, Uwe Schindler,
Mark Miller)

* SOLR-9037: Replace multiple "/replication" strings with one static constant.

(Christine Poerschke)

* SOLR-9047: zkcli should allow alternative locations for log4j configuration

(Gregory Chanan)

* SOLR-9066: Make CountMetric return long instead of double (Kevin Risden)

* SOLR-9065, SOLR-9072, SOLR-9132: Migrate some distributed tests to

SolrCloudTestCase. (Alan Woodward)

* SOLR-8184: Negative tests for JDBC Connection String (Susheel Kumar, Jason
Gerlowski, Kevin Risden)

* SOLR-8458: Add Streaming Expressions tests for parameter substitution (Joel

Bernstein, Cao Manh Dat, Dennis Gove, Kevin Risden)

* SOLR-8467: CloudSolrStream and FacetStream should take a SolrParams object rather

than a
 Map<String, String> to allow more complex Solr queries to be specified. (Erick
Erickson)

* SOLR-9083: Remove all <types> and <fields> from schemas. NOTE: as in the JIRA I
left a few in to insure the (no cost)
back compat. (Erick Erickson)

* SOLR-9105: Fix a bunch of typos across 103 files (Bartosz Krasiński via janhoy)

* SOLR-9159: New cloud based concurrent atomic update test (hossman)

* SOLR-9119: several static methods in ValueSourceParser have been made private

(hossman)

* SOLR-9110: Move JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib

Tests to SolrCloudTestCase (Mikhail Khludnev)

* SOLR-9136: Separate out the error statistics into server-side error vs client-
side error
(Jessica Cheng Mallet via Erick Erickson)

* SOLR-9107: new @RandomizeSSL annotation for more fine grained control of SSL
testing (hossman, sarowe)

* SOLR-9081: Make SolrTestCaseJ4.beforeClass() / .afterClass() public so it

works with Mockito (Georg Sorst, Alan Woodward)

* SOLR-8445: fix line separator in log4j.properties files (Ahmet Arslan via Mikhail
Khludnev)

* SOLR-2199: DataImportHandler (DIH) JdbcDataSource supports multiple resultsets

per query (Kristine Jetzke, Mark Waddle via Mikhail Khludnev)

================== 6.0.1 ==================

Upgrade Notes
----------------------

* If you use historical dates, specifically on or before the year 1582, you should
re-index.

Bug Fixes
----------------------

* SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe. (Scott

Blum, hoss,
sarowe, Erick Erickson, Mark Miller, shalin)

* SOLR-9016: Fix SolrIdentifierValidator to not allow empty identifiers. (Shai

Erera)

* SOLR-8992: Restore Schema API GET method functionality removed in 6.0 (noble,
Steve Rowe)

* SOLR-9080, SOLR-9085: (6.0 bug) For years <= 1582, date math (round,add,sub)
introduced error. Range faceting
on such dates was also affected. With this fixed, this is the first release
range faceting works on BC years.
(David Smiley)
* SOLR-8857: HdfsUpdateLog does not use configured or new default number of version
buckets and is
hard coded to 256. (Mark Miller, yonik, Gregory Chanan)

* SOLR-8902: Make sure ReturnFields only returns the requested fields from (fl=)
evn when
DocumentTransformers ask for getExtraRequestFields() (ryan)

* SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting

in NPE.
(Scott Blum via David Smiley)

* SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes.
(David Smiley)

* SOLR-9004: Fix "name" field type definition in films example. (Alexandre

Rafalovitch via Varun Thacker)

* SOLR-8990: Fix top term links from schema browser page to use {!term} parser
(hossman)

* SOLR-8971: Preserve root cause when wrapping exceptions (hossman)

* SOLR-9034: Atomic updates failed to work when there were copyField targets that
had docValues
enabled. (Karthik Ramachandran, Ishan Chattopadhyaya, yonik)

* SOLR-9028: Fixed some test related bugs preventing SSL + ClientAuth from ever
being tested (hossman)

* SOLR-9059: NPE in SolrClientCache following collection reload (Joel Bernstein,

Ryan Yacyshyn)

* SOLR-8792: ZooKeeper ACL support fixed. (Esther Quansah, Ishan Chattopadhyaya,

Steve Rowe)

* SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter.

(Scott Blum, Mark Miller, shalin)

* SOLR-9036: Solr slave is doing full replication (entire index) of index after
master restart.
(Lior Sapir, Mark Miller, shalin)

* SOLR-9058: Makes HashJoinStream and OuterHashJoinStream support different field

names in the
incoming streams, eg. fieldA=fieldB. (Dennis Gove, Stephan Osthold)

* SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor.

(Christine Poerschke)

* SOLR-9118: HashQParserPlugin should trim partition keys (Joel Bernstein)

* SOLR-9117: The first SolrCore is leaked after reload. (Jessica Cheng Mallet via
shalin)

* SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore

is reloaded.
(Jessica Cheng Mallet via shalin)
* SOLR-8801: /bin/solr create script always returns exit code 0 when a
collection/core already exists.
(Khalid Alharbi, Marius Grama via Steve Rowe)

* SOLR-9134: Fix RestManager.addManagedResource return value. (Christine Poerschke)

Other Changes
----------------------
* SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the
single-usage policy.
(Jason Gerlowski, Benoit Vanalderweireldt, shalin)

* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the
Replication tab won't be displayed
anymore. The Replication tab is only beneficial to users running Solr in master-
slave mode. (Varun Thacker)

* SOLR-8985: Added back support for 'includeDynamic' flag to /schema/fields

endpoint (noble)

* SOLR-9131: Fix "start solr" text in cluster.vm Velocity template (janhoy)

* SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability

(Jeff Field, Mike Drob via janhoy)

* SOLR-9115: Get rid of javax.xml.bind.DatatypeConverter in SimplePostTool

for Java 9 compatibility. (Uwe Schindler)

* SOLR-5776,SOLR-9068,SOLR-8970:
- Refactor SSLConfig so that SSLTestConfig can provide SSLContexts using a
NullSecureRandom
to prevent SSL tests from blocking on entropy starved machines.
- SSLTestConfig: Alternate (pseudo random) NullSecureRandom for Constants.SUN_OS.
- SSLTestConfig: Replace NullSecureRandom w/ NotSecurePseudoRandom.
- Change SSLTestConfig to use a keystore file that is included as a resource in
the
test-framework jar so users subclassing SolrTestCaseJ4 don't need to preserve
magic paths.
(hossman)

================== 6.0.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.12.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.8.v20160314

System Requirements
----------------------
* LUCENE-5950: Move to Java 8 as minimum Java version.
(Ryan Ernst, Uwe Schindler)
Upgrading from Solr 5.x
----------------------

* The deprecated SolrServer and subclasses have been removed, use SolrClient
instead.

* The deprecated <nrtMode> configuration in solrconfig.xml has been removed.

Please remove it from solrconfig.xml.

* SolrClient.shutdown() has been removed, use SolrClient.close() instead.

* The deprecated zkCredientialsProvider element in solrcloud section of solr.xml

is now removed. Use the correct spelling (zkCredentialsProvider) instead.

* SOLR-7957: internal/expert - ResultContext was significantly changed and expanded

to allow for multiple full query results (DocLists) per Solr request.
TransformContext was rendered redundant and was removed. (yonik)

* Several changes have been made regarding the "Similarity" used in Solr, in order
to provide
better default behavior for new users. There are 3 key impacts of these changes
on existing
users who upgrade:
* DefaultSimilarityFactory has been removed. If you currently have
DefaultSimilarityFactory explicitly
referenced in your schema.xml, edit your config to use the functionally
identical
ClassicSimilarityFactory. See SOLR-8239 for more details.
* The implicit default Similarity used when no <similarity/> is configured in
schema.xml has
been changed to SchemaSimilarityFactory. Users who wish to preserve back-
compatible behavior should
either explicitly configure ClassicSimilarityFactory, or ensure that the
luceneMatchVersion
for the collection is less then 6.0. See SOLR-8270 + SOLR-8271 for details.
* SchemaSimilarityFactory has been modified to use BM25Similarity as the default
for fieldTypes that
do not explicitly declare a Similarity. The legacy behavior of using
ClassicSimilarity as the
default will occur if the luceneMatchVersion for the collection is less then
6.0, or the
'defaultSimFromFieldType' configuration option may be used to specify any
default of your choosing.
See SOLR-8261 + SOLR-8329 for more details.

* If your solrconfig.xml file doesn't explicitly mention the schemaFactory to use

then Solr will choose
the ManagedIndexSchemaFactory by default. Previously it would have chosen
ClassicIndexSchemaFactory.
This means that the Schema APIs ( /<collection>/schema ) are enabled and the
schema is mutable.
When Solr starts your schema.xml file will be renamed to managed-schema. If you
want to retain the old behaviour
then please ensure that the solrconfig.xml explicitly uses the
ClassicIndexSchemaFactory :
<schemaFactory class="ClassicIndexSchemaFactory"/> or your luceneMatchVersion in
the solrconfig.xml is less than 6.0
* SolrIndexSearcher.QueryCommand and QueryResult were moved to their own classes.
If you reference them
in your code, you should import them under o.a.s.search (or use your IDE's
"Organize Imports").

* SOLR-8698: 'useParams' attribute specified in request handler cannot be

overridden from request params

* When requesting stats in date fields, "sum" is now a double value instead of a
date. See SOLR-8671

* SOLR-8736: The deprecated GET methods for schema are now accessible through the
bulk API. The output
has less details and is not backward compatible.

* In the past, Solr guaranteed that retrieval of multi-valued fields would preserve
the order of values.
Because values may now be retrieved from column-stored fields (docValues="true"),
in conjunction with the
fact that docValues do not currently preserve order, means that users should set
useDocValuesAsStored="false"
to prevent future optizations from using the column-stored values over the row-
stored values when
fields have both stored="true" and docValues="true".

* Formatted date-times from Solr have some differences. If the year is more than 4
digits, there is a leading '+'.
When there is a non-zero number of milliseconds, it is padded with zeros to 3
digits. Negative year (BC) dates are
now possible. Parsing: It is now an error to supply a portion of the date out of
its, range, like 67 seconds.

* SolrJ no longer includes DateUtil. If for some reason you need to format or parse
dates, simply use Instant.format()
and Instant.parse().

* If you are using an RPT or other spatial field referencing Spatial4j in its
configuration, then replace the string
"com.spatial4j.core" with "org.locationtech.spatial4j". Consider updating direct
to Solr 6.2 which accepts the
old value, albeit with a warning.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-3085: New edismax param mm.autoRelax which helps in certain cases of the
stopwords/zero-hits issue (janhoy)

* SOLR-7560: Parallel SQL Support (Joel Bernstein)

* SOLR-7707: Add StreamExpression Support to RollupStream (Dennis Gove, Joel

Bernstein)

* SOLR-7903: Add the FacetStream to the Streaming API and wire it into the
SQLHandler (Joel Bernstein)
* SOLR-7986: JDBC Driver for SQL Interface (Uwe Schindler, Joel Bernstein)

* SOLR-8038: Add the StatsStream to the Streaming API and wire it into the
SQLHandler (Joel Bernstein)

* SOLR-8086: Add support for SELECT DISTINCT queries to the SQL interface (Joel
Bernstein)

* SOLR-7543: Basic graph traversal query

Example: {!graph from="node_id" to="edge_id"}id:doc_1
(Kevin Watters, yonik)

* SOLR-6273: Cross Data Center Replication. Active/passive replication for separate

SolrClouds hosted on separate data centers. (Renaud Delbru, Yonik Seeley via
Erick Erickson)

* SOLR-7938: MergeStream now supports merging more than 2 streams together (Dennis
Gove)

* SOLR-8198: Change ReducerStream to use StreamEqualitor instead of

StreamComparator (Dennis Gove)

* SOLR-8268: StatsStream now implements the Expressible interface (Dennis Gove)

* SOLR-7584: Adds Inner and LeftOuter Joins to the Streaming API and Streaming
Expressions (Dennis Gove, Corey Wu)

* SOLR-8188: Adds Hash and OuterHash Joins to the Streaming API and Streaming
Expressions (Dennis Gove)

* SOLR-7669: Add SelectStream and Tuple Operations to the Streaming API and
Streaming Expressions (Dennis Gove)

* SOLR-8337: Add ReduceOperation and wire it into the ReducerStream (Joel

Bernstein)

* SOLR-7904: Add StreamExpression Support to FacetStream (Dennis Gove)

* SOLR-6398: Add IterativeMergeStrategy to support running Parallel Iterative

Algorithms inside of Solr
(Joel Bernstein)

* SOLR-8436: Real-time get now supports filters. (yonik)

* SOLR-7535: Add UpdateStream to Streaming API and Streaming Expression (Jason

Gerlowski, Joel Bernstein)

* SOLR-8479: Add JDBCStream to Streaming API and Streaming Expressions for

integration with external data sources
(Dennis Gove)

* SOLR-8002: Add column alias support to the Parallel SQL Interface (Joel
Bernstein)

* SOLR-7525: Add ComplementStream and IntersectStream to the Streaming API and

Streaming Expressions
(Dennis Gove, Jason Gerlowski, Joel Bernstein)

* SOLR-8415: Provide command to switch between non/secure mode in ZK

 (Mike Drob, Gregory Chanan)

* SOLR-8556: Add ConcatOperation to be used with the SelectStream (Joel Bernstein,

Dennis Gove)

* SOLR-8550: Add asynchronous DaemonStreams to the Streaming API (Joel Bernstein)

* SOLR-8285: Ensure the /export handler works with NULL field values (Joel
Bernstein)

* SOLR-8502: Improve Solr JDBC Driver to support SQL Clients like DBVisualizer
(Kevin Risden, Joel Bernstein)

* SOLR-8588: Add TopicStream to the streaming API to support publish/subscribe

messaging (Joel Bernstein, Kevin Risden)

* SOLR-8666: Adds header 'zkConnected' to response of SearchHandler and

PingRequestHandler to notify the client when
a connection to zookeeper has been lost and there is a possibility of stale data
on the node the request is coming
from. (Keith Laban, Dennis Gove)

* SOLR-8522: Make it possible to use ip fragments in replica placement rules , such

as ip_1, ip_2 etc (Arcadius Ahouansou, noble)

* SOLR-8698: params.json can now specify 'appends' and 'invariants' (noble)

* SOLR-8831: allow _version_ field to be retrievable via docValues (yonik)

Bug Fixes
----------------------
* SOLR-8386: Add field option in the new admin UI schema page loads up even when no
schemaFactory has been
explicitly specified since the default is ManagedIndexSchemaFactory. (Erick
Erickson, Upayavira, Varun Thacker)

* SOLR-8191: Guard against CloudSolrStream close method NullPointerException

(Kevin Risden, Joel Bernstein)

* SOLR-8485: SelectStream now properly handles non-lowercase and/or quoted select

field names (Dennis Gove)

* SOLR-8525: Fix a few places that were failing to pass dimensional

values settings when copying a FieldInfo (Ishan Chattopadhyaya via
Mike McCandless)

* SOLR-8409: Ensures that quotes in solr params (eg. q param) are properly handled
(Dennis Gove)

* SOLR-8640: CloudSolrClient does not send credentials for update request (noble,
hoss)

* SOLR-8461: CloudSolrStream and ParallelStream can choose replicas that are not
active
(Cao Manh Dat, Varun Thacker, Joel Bernstein)

* SOLR-8527: Improve JdbcTest to cleanup properly on failures (Kevin Risden, Joel

Bernstein)
* SOLR-8578: Successful or not, requests are not always fully consumed by Solrj
clients and we
count on HttpClient or the JVM. (Mark Miller)

* SOLR-8683: Always consume the full request on the server, not just in the case of
an error.
(Mark Miller)

* SOLR-8416: The collections create API should return after all replicas are
active.
(Michael Sun, Mark Miller, Alexey Serba)

* SOLR-8701: CloudSolrClient decides that there are no healthy nodes to handle a

request too early.
(Mark Miller)

* SOLR-8694: DistributedMap/Queue can create too many Watchers and some code
simplification.
(Scott Blum via Mark Miller)

* SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection
events and
make this handling more consistent. (Scott Blum via Mark Miller)

* SOLR-8633: DistributedUpdateProcess processCommit/deleteByQuery call finish on

DUP and
SolrCmdDistributor, which violates the lifecycle and can cause bugs. (hossman via
Mark Miller)

* SOLR-8656: PeerSync should use same nUpdates everywhere. (Ramsey Haddad via Mark
Miller)

* SOLR-8697, SOLR-8837: Scope ZK election nodes by session to prevent elections

from interfering with each other
and other small LeaderElector improvements. (Scott Blum via Mark Miller, Alan
Woodward)

* SOLR-8599: After a failed connection during construction of SolrZkClient attempt

to retry until a connection
can be made. (Keith Laban, Dennis Gove)

* SOLR-8497: Merge index does not mark the Directory objects it creates as 'done'
and they are retained in the
Directory cache. (Sivlio Sanchez, Mark Miller)

* SOLR-8696: Start the Overseer before actions that need the overseer on init and
when reconnecting after
zk expiration and improve init logic. (Scott Blum, Mark Miller)

* SOLR-8420: Fix long overflow in sumOfSquares for Date statistics. (Tom Hill,
Christine Poerschke,
Tomás Fernández Löbbe)

* SOLR-8748: OverseerTaskProcessor limits number of concurrent tasks to just 10

even though the thread pool
size is 100. The limit has now been increased to 100. (Scott Blum, shalin)

* SOLR-8375: ReplicaAssigner rejects valid nodes (Kelvin Tan, noble)

* SOLR-8738: Fixed false success response when invalid deleteByQuery requests
intially hit non-leader
cloud nodes (hossman)

* SOLR-8771: Multi-threaded core shutdown creates executor per core. (Mike Drob via
Mark Miller)

* SOLR-8145: Fix position of OOM killer script when starting Solr in the background
(Jurian Broertjes via
Timothy Potter)

* SOLR-8769: Fix document exclusion in mlt query parser in Cloud mode for schemas
that have non-"id"
unique field (Erik Hatcher, Anshum Gupta)

* SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only
participating in replica
placement. splitshard should preassign nodes using rules, if rules are present
(noble, Shai Erera)

* SOLR-8779: Fix missing InterruptedException handling in ZkStateReader.java (Varun

Thacker)

* SOLR-8449: Fix the core restore functionality to allow restoring multiple times
on the same core
(Johannes Brucher, Varun Thacker)

* SOLR-8155: JSON Facet API - field faceting on a multi-valued string field without
docValues (i.e. UnInvertedField implementation), but with a prefix or with a sort
other than count, resulted in incorrect results. This has been fixed, and
facet.prefix
support for facet.method=uif has been enabled. (Mikhail Khludnev, yonik)

* SOLR-8790: Collections API responses contain node name in the core-level

responses that are
returned. (Anshum Gupta)

* SOLR-8804: Fix a race condition in the ClusterStatus API call whereby the call
would fail when a concurrent delete
collection api command was executed (Alexey Serba, Varun Thacker)

* SOLR-8835: JSON Facet API: fix faceting exception on multi-valued numeric fields
that
have docValues. (yonik)

* SOLR-8838: Returning non-stored docValues is incorrect for negative floats and

doubles.
(Ishan Chattopadhyaya, Steve Rowe)

* SOLR-8867: {!frange} queries will now avoid matching documents without a value in
the
numeric field. For more complex functions, FunctionValues.exists() must also
return true
for the document to match. (yonik)

* SOLR-8886: Fix TrieField.toObject(IndexableField) to work for field with

docValues
enabled. (yonik)
* SOLR-8891: Fix StrField.toObject and toExternal to work with docValue
IndexableField
instances. (yonik)

* SOLR-8865: Real-time get sometimes fails to retrieve stored fields from

docValues.
(Ishan Chattopadhyaya, yonik)

Optimizations
----------------------
* SOLR-7876: Speed up queries and operations that use many terms when timeAllowed
has not been
specified. Speedups of up to 8% were observed. (yonik)

* SOLR-8037: Speed up creation of filters from term range queries (i.e. non-numeric
range queries)
and use the filter cache for term range queries that are part of larger queries.
Some observed
speedups were up to 2.5x for production of filters, and up to 10x for query
evaluation with
embedded term range queres that resulted in filter cache hits. (yonik)

* SOLR-8559: FCS facet performance optimization which significantly speeds up

processing when terms
are high cardinality and the matching docset is small. When facet minCount > 0
and the number of
matching documents is small (or 0) this enhancement prevents considering terms
which have a 0
count. Also includes change to move to the next non-zero term value when
selecting a segment
position. (Keith Laban, Steve Bower, Dennis Gove)

* SOLR-8532: Optimize GraphQuery when maxDepth is set by not collecting edges at

the maxDepth level.
(Kevin Watters via yonik)

* SOLR-8669: Non binary responses use chunked encoding because we flush the
outputstream early.
(Mark Miller)

* SOLR-8720: ZkController#publishAndWaitForDownStates should use

#publishNodeAsDown. (Mark Miller)

* SOLR-8082: Can't query against negative float or double values when

indexed="false"
docValues="true" multiValued="false". (hossman, Ishan Chattopadhyaya, yonik,
Steve Rowe)

Other Changes
----------------------

* SOLR-6127: Improve example docs, using films data (Varun Thacker via ehatcher)

* SOLR-6895: Deprecated SolrServer classes have been removed (Alan Woodward,

Erik Hatcher)

* SOLR-6954: Deprecated SolrClient.shutdown() method removed (Alan Woodward)

* SOLR-7355: Switch from Google's ConcurrentLinkedHashMap to Caffeine. Only
affects HDFS support. (Ben Manes via Shawn Heisey)

* SOLR-7624: Remove deprecated zkCredientialsProvider element in solrcloud section

of solr.xml.
(Xu Zhang, Per Steffensen, Ramkumar Aiyengar, Mark Miller)

* SOLR-7513: Add Equalitors to Streaming Expressions (Dennis Gove, Joel Bernstein)

* SOLR-7528: Simplify Interfaces used in Streaming Expressions (Dennis Gove, Joel

Bernstein)

* SOLR-7554: Add checks in Streams for incoming stream order (Dennis Gove, Joel
Bernstein)

* SOLR-7441: Improve overall robustness of the Streaming stack: Streaming API,

Streaming Expressions, Parallel SQL (Joel Bernstein)

* SOLR-8153: Support upper case and mixed case column identifiers in the SQL
interface
(Joel Bernstein)

* SOLR-8132: HDFSDirectoryFactory now defaults to using the global block cache.

(Mark Miller)

* SOLR-8261: Change SchemaSimilarityFactory default to BM25Similarity (hossman)

* SOLR-8259: Remove deprecated JettySolrRunner.getDispatchFilter()

* SOLR-8258: Change default hdfs tlog replication factor from 1 to 3. (Mark Miller)

* SOLR-8270: Change implicit default Similarity to use BM25 when luceneMatchVersion

>= 6 (hossman)

* SOLR-8271: Change implicit default Similarity to use SchemaSimilarityFactory when

luceneMatchVersion >= 6 (hossman)

* SOLR-8179: SQL JDBC - DriverImpl loadParams doesn't support keys with no values
in the connection string
(Kevin Risden, Joel Bernstein)

* SOLR-8131: Make ManagedIndexSchemaFactory the default schemaFactory when

luceneMatchVersion >= 6
(Uwe Schindler, shalin, Varun Thacker)

* SOLR-8266: Remove Java Serialization from the Streaming API. The /stream handler
now only accepts
Streaming Expressions. (Jason Gerlowski, Joel Bernstein)

* SOLR-8426: Enable /export, /stream and /sql handlers by default and remove them
from example configs. (shalin)

* SOLR-8443: Change /stream handler http param from "stream" to "expr" (Joel
Bernstein, Dennis Gove)

* SOLR-5209: Unloading or deleting the last replica of a shard now no longer

cascades to remove the shard from the clusterstate. (Christine Poerschke)

* SOLR-8190: Implement Closeable on TupleStream (Kevin Risden, Joel Bernstein)

* SOLR-8529: Improve JdbcTest to not use plain assert statements (Kevin Risden,
Joel Bernstein)

* SOLR-7339: Upgrade Jetty to v9.3.8.v20160314. (Gregg Donovan, shalin, Mark

Miller, Steve Rowe)

* SOLR-5730: Make Lucene's SortingMergePolicy and EarlyTerminatingSortingCollector

configurable in Solr.
(Christine Poerschke, hossmann, Tomás Fernández Löbbe, Shai Erera)

* SOLR-8677: Prevent shards containing invalid characters from being created.

Checks added server-side
and in SolrJ. (Shai Erera, Jason Gerlowski, Anshum Gupta)

* SOLR-8693: Improve ZkStateReader logging. (Scott Blum via Mark Miller)

* SOLR-8710: Upgrade morfologik-stemming to version 2.1.0. (Dawid Weiss)

* SOLR-8711: Upgrade Carrot2 clustering dependency to 3.12.0. (Dawid Weiss)

* SOLR-8690: Make peersync fingerprinting optional with solr.disableFingerprint

system
property. (yonik)

* SOLR-8691: Cache index fingerprints per searcher. (yonik)

* SOLR-8746: Renamed Overseer.getInQueue to getStateUpdateQueue, getInternalQueue

to getInternalWorkQueue
and added javadocs. (Scott Blum, shalin)

* SOLR-8752: Add a test for SizeLimitedDistributedMap and improve javadocs.

(shalin)

* SOLR-8671: Date statistics: make "sum" a double instead of a long/date (Tom Hill,
Christine Poerschke,
Tomás Fernández Löbbe)

* SOLR-8713: new UI and example solrconfig files point to Reference Guide for Solr
Query Syntax instead
of the wiki. (Marius Grama via Tomás Fernández Löbbe)

* SOLR-8758: Add a new SolrCloudTestCase class, using MiniSolrCloudCluster (Alan

Woodward)

* SOLR-8736: schema GET operations on fields, dynamicFields, fieldTypes, copyField

are
reimplemented as a part of the bulk API with less details (noble)

* SOLR-8766: Remove deprecated <admin> tag in solrconfig.xml and support for

admin/gettableFiles
(noble, Jason Gerlowski, Varun Thacker)

* SOLR-8799: Improve error message when tuple can't be read by SolrJ JDBC (Kevin
Risden, Joel Bernstein)

* SOLR-8836: Return 400, and a SolrException when an invalid json is provided to

the update handler
instead of 500. (Jason Gerlowski via Anshum Gupta)
* SOLR-8740: docValues are now enabled by default for most non-text (string, date,
and numeric) fields
in the schema templates. (yonik)

* SOLR-8819: Implement DatabaseMetaDataImpl getTables() and fix getSchemas(). (Trey

Cahill,
Joel Bernstein, Kevin Risden)

* SOLR-8810: Implement Connection.setReadOnly, Statement.set/getFetchSize,

ResultSet.getType (Kevin Risden)

* SOLR-8904: All dates are formatted via Instant.toString() (ISO-8601); see Solr
upgrade notes for differences. Will
now parse (and format) dates with a leading '+' or '-' (BC dates or dates > 4
digit year.
[value] and ms() and contrib/analytics now parse with date math. (David Smiley)

* SOLR-8904: DateUtil in SolrJ moved to the extraction contrib as

ExtractionDateUtil. Obsolete methods were removed.
(David Smiley)

======================= 5.5.5 =======================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.10.4
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.2.13.v20150730

Upgrade Notes
----------------------

* SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... })

the resolving of external entities is now disallowed by default.

* SOLR-11482: RunExecutableListener was deprecated and is disabled by default for

security reasons. Legacy applications still using it must explicitely pass
'-Dsolr.enableRunExecutableListener=true' to the Solr command line.
Be aware that you should really disable API-based config editing at the same
time, using '-Ddisable.configEdit=true'! (Uwe Schindler)

Bug Fixes
----------------------

* SOLR-10420: Leaking one SolrZkClient instance per second (Scott Blum, Cao Manh
Dat, Markus Jelsma, Steve Rowe)

* SOLR-11477: Disallow resolving of external entities in the XML query parser

(defType=xmlparser).
(Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke)

Other
----------------------

* SOLR-11486: Update jmatio to 1.2 and prevent deserialization inside Matlab files.
(Uwe Schindler, Steve Rowe)

======================= 5.5.4 =======================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.15.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.14.v20161028

Other Changes
----------------------

* SOLR-9819: Upgrade commons-fileupload to 1.3.2, fixing a potential vulnerability

CVE-2016-3092 (Anshum Gupta)

* SOLR-10031: Validation of filename params in ReplicationHandler (Hrishikesh

Gadre, janhoy)

======================= 5.5.3 =======================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.13
Carrot2 3.12.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.3.8.v20160314

(No Changes)

======================= 5.5.2 =======================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.10.4
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.2.13.v20150730

Bug Fixes
---------------------

* SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection
events and
make this handling more consistent. (Scott Blum via Mark Miller)

* SOLR-9198: config APIs unable to add multiple values with same name (noble)

* SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed (Scott Blum, Noble Paul)

* SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided

and there are explicit operators (except for AND) - addresses problems caused by
SOLR-2649.
(Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe)

* SOLR-9034: Atomic updates failed to work when there were copyField targets that
had docValues
enabled. (Karthik Ramachandran, Ishan Chattopadhyaya, yonik)

* SOLR-8940: Fix group.sort option (hossman)

* SOLR-8857: HdfsUpdateLog does not use configured or new default number of version
buckets and is
hard coded to 256. (Mark Miller, yonik, Gregory Chanan)

* SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting

in NPE.
(Scott Blum via David Smiley)

* SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes.
(David Smiley)

* SOLR-9004: Fix "name" field type definition in films example. (Alexandre

Rafalovitch via Varun Thacker)

* SOLR-8990: Fix top term links from schema browser page to use {!term} parser
(hossman)

* SOLR-8971: Preserve root cause when wrapping exceptions (hossman)

* SOLR-8792: ZooKeeper ACL support fixed. (Esther Quansah, Ishan Chattopadhyaya,

Steve Rowe)

* SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter.

(Scott Blum, Mark Miller, shalin)

* SOLR-9036: Solr slave is doing full replication (entire index) of index after
master restart.
(Lior Sapir, Mark Miller, shalin)

* SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor.

(Christine Poerschke)

* SOLR-9118: HashQParserPlugin should trim partition keys (Joel Bernstein)

* SOLR-9117: The first SolrCore is leaked after reload. (Jessica Cheng Mallet via
shalin)

* SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore

is reloaded.
(Jessica Cheng Mallet via shalin)

* SOLR-8801: /bin/solr create script always returns exit code 0 when a

collection/core already exists.
(Khalid Alharbi, Marius Grama via Steve Rowe)

* SOLR-9134: Fix RestManager.addManagedResource return value. (Christine Poerschke)

* SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD
(janhoy)

* SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used

with "cursorMark".
(James Dyer)

* SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH)

(Kristine Jetzke via Mikhail Khludnev)

* SOLR-8676: keep LOG4J_CONFIG in solr.cmd (Kristine Jetzke via Mikhail Khludnev)

* SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to

FCS, causing slowdowns (Alessandro Benedetti, Jesse McLaughlin, Alan Woodward)

Other Changes
----------------------

* SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the

single-usage policy.
(Jason Gerlowski, Benoit Vanalderweireldt, shalin)

* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the
Replication tab won't be displayed
anymore. The Replication tab is only beneficial to users running Solr in master-
slave mode. (Varun Thacker)

* SOLR-9131: Fix "start solr" text in cluster.vm Velocity template (janhoy)

* SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability

(Jeff Field, Mike Drob via janhoy)

* SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to

serialize a value.
(David Smiley)

* SOLR-8933: Solr should not close container streams. (Mike Drob, Uwe Schindler,
Mark Miller)

* SOLR-9037: Replace multiple "/replication" strings with one static constant.

(Christine Poerschke)

* SOLR-9047: zkcli should allow alternative locations for log4j configuration

(Gregory Chanan)

* SOLR-9105: Fix a bunch of typos across 103 files (Bartosz Krasiński via janhoy)
* SOLR-8445: fix line separator in log4j.properties files (Ahmet Arslan via Mikhail
Khludnev)

* SOLR-8674: Stop ignoring sysprop solr.tests.mergePolicy, and make tests randomly

choose between
setting <mergePolicy> and <mergePolicyFactory>, which was added in SOLR-8621.
(Christine Poerschke)

======================= 5.5.1 =======================

Bug Fixes
----------------------

* SOLR-8737: Managed synonym lists do not include the original term in the expand
(janhoy)

* SOLR-8734: fix (maxMergeDocs|mergeFactor) deprecation warnings: in solrconfig.xml

<maxMergeDocs|mergeFactor> may not be combined with <mergePolicyFactory> and
<maxMergeDocs|mergeFactor> on their own or combined with <mergePolicy> is a
warning.
(Christine Poerschke, Shai Erera)

* SOLR-8712: Variable solr.core.instanceDir was not being resolved (Kristine

Jetzke, Shawn Heisey, Alan Woodward)

* SOLR-8793: Fix Core admin status API to not fail when computing the size of the
segments_N
file if the file no longer exists (for example, if a commit happened and the
IndexReader
hasn't refreshed yet). In this case the reported size of the file is -1.
(Shai Erera, Alexey Serba, Richard Coggins)

* SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only
participating in replica
placement. splitshard should preassign nodes using rules, if rules are present
(noble, Shai Erera)

* SOLR-8838: Returning non-stored docValues is incorrect for negative floats and

doubles.
(Ishan Chattopadhyaya, Steve Rowe)

* SOLR-8870: AngularJS Query tab no longer URL-encodes the /select part of the
request, fixing possible 404 issue
when Solr is behind a proxy. Also, now supports old-style &qt param when handler
not prefixed with "/" (janhoy)

* SOLR-8725: Allow hyphen in collection, core, shard, and alias name as the non-
first character
(Anshum Gupta) (from 6.0)

* SOLR-8155: JSON Facet API - field faceting on a multi-valued string field without
docValues (i.e. UnInvertedField implementation), but with a prefix or with a sort
other than count, resulted in incorrect results. This has been fixed, and
facet.prefix
support for facet.method=uif has been enabled. (Mikhail Khludnev, yonik)

* SOLR-8835: JSON Facet API: fix faceting exception on multi-valued numeric fields
that
 have docValues. (yonik)

* SOLR-8908: Fix to OnReconnect listener registration to allow listeners to

deregister, such
as when a core is reloaded or deleted to avoid a memory leak. (Timothy Potter)

* SOLR-8662: SchemaManager waits correctly for replicas to be notified of a new

change
(sarowe, Noble Paul, Varun Thacker)

* SOLR-9007: Remove mention of the managed_schema_configs as valid config directory

when creating
the collection for the SolrCloud example. (Timothy Potter)

* SOLR-8983: Cleanup clusterstate and replicas for a failed create collection

request
(Varun Thacker, Anshum Gupta)

* SOLR-8578: Successful or not, requests are not always fully consumed by Solrj
clients and we
count on HttpClient or the JVM. (Mark Miller)

* SOLR-8683: Always consume the full request on the server, not just in the case of
an error.
(Mark Miller)

* SOLR-8701: CloudSolrClient decides that there are no healthy nodes to handle a

request too early.
(Mark Miller)

* SOLR-8416: The collections create API should return after all replicas are
active.
(Michael Sun, Mark Miller, Alexey Serba)

* SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe. (Scott

Blum, hoss,
sarowe, Erick Erickson, Mark Miller, shalin)

* SOLR-8973: Zookeeper frenzy when a core is first created. (Janmejay Singh, Scott
Blum, shalin)

* SOLR-8694: DistributedMap/Queue can create too many Watchers and some code
simplification.
(Scott Blum via Mark Miller)

* SOLR-8633: DistributedUpdateProcess processCommit/deleteByQuery call finish on

DUP and
SolrCmdDistributor, which violates the lifecycle and can cause bugs. (hossman via
Mark Miller)

* SOLR-8656: PeerSync should use same nUpdates everywhere. (Ramsey Haddad via Mark
Miller)

* SOLR-8697, SOLR-8837: Scope ZK election nodes by session to prevent elections

from interfering with each other
and other small LeaderElector improvements. (Scott Blum via Mark Miller, Alan
Woodward)

* SOLR-8599: After a failed connection during construction of SolrZkClient attempt

to retry until a connection
can be made. (Keith Laban, Dennis Gove)

* SOLR-8420: Fix long overflow in sumOfSquares for Date statistics. (Tom Hill,
Christine Poerschke,
Tomás Fernández Löbbe)

* SOLR-8748: OverseerTaskProcessor limits number of concurrent tasks to just 10

even though the thread pool
size is 100. The limit has now been increased to 100. (Scott Blum, shalin)

* SOLR-8375: ReplicaAssigner rejects valid nodes (Kelvin Tan, noble)

* SOLR-8738: Fixed false success response when invalid deleteByQuery requests

intially hit non-leader
cloud nodes (hossman)

* SOLR-8771: Multi-threaded core shutdown creates executor per core. (Mike Drob via
Mark Miller)

* SOLR-8145: Fix position of OOM killer script when starting Solr in the background
(Jurian Broertjes via
Timothy Potter)

* SOLR-8769: Fix document exclusion in mlt query parser in Cloud mode for schemas
that have non-"id"
unique field (Erik Hatcher, Anshum Gupta)

* SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only
participating in replica
placement. splitshard should preassign nodes using rules, if rules are present
(noble, Shai Erera)

* SOLR-8779: Fix missing InterruptedException handling in ZkStateReader.java (Varun

Thacker)

* SOLR-8449: Fix the core restore functionality to allow restoring multiple times
on the same core
(Johannes Brucher, Varun Thacker)

* SOLR-8790: Collections API responses contain node name in the core-level

responses that are
returned. (Anshum Gupta)

* SOLR-8804: Fix a race condition in the ClusterStatus API call whereby the call
would fail when a concurrent delete
collection api command was executed (Alexey Serba, Varun Thacker)

* SOLR-9016: Fix SolrIdentifierValidator to not allow empty identifiers. (Shai

Erera)

* SOLR-8886: Fix TrieField.toObject(IndexableField) to work for field with

docValues
enabled. (yonik)

* SOLR-8891: Fix StrField.toObject and toExternal to work with docValue

IndexableField
instances. (yonik)
* SOLR-8865: Real-time get sometimes fails to retrieve stored fields from
docValues.
(Ishan Chattopadhyaya, yonik)

* SOLR-9046: Fix solr.cmd that wrongly assumes Jetty will always listen on 0.0.0.0.
(Bram Van Dam, Uwe Schindler)

Other Changes
----------------------

* SOLR-8758: Add a new SolrCloudTestCase class, using MiniSolrCloudCluster (Alan

Woodward)

======================= 5.5.0 =======================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.10.4
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.2.13.v20150730

Upgrading from Solr 5.4

-----------------------

* The Solr schema version has been increased to 1.6. Since schema version 1.6, all
non-stored docValues fields
will be returned along with other stored fields when all fields (or pattern
matching globs) are specified
to be returned (e.g. fl=*) for search queries. This behavior can be turned on and
off by setting
'useDocValuesAsStored' parameter for a field or a field type to true (default
since schema version 1.6)
or false (default till schema version 1.5).
Note that enabling this property has performance implications because DocValues
are column-oriented and may
therefore incur additional cost to retrieve for each returned document. All
example schema are upgraded to
version 1.6 but any older schemas will default to useDocValuesAsStored=false and
continue to work as in
older versions of Solr. If this new behavior is desirable, then you should set
version attribute in your
schema file to '1.6'. Re-indexing is not necessary to upgrade the schema version.
Also note that while returning non-stored fields from docValues (default in
schema versions 1.6+, unless
useDocValuesAsStored is false), the values of a multi-valued field are returned
in sorted order.
If you require the multi-valued fields to be returned in the original insertion
order, then make your
multi-valued field as stored. This requires re-indexing.
See SOLR-8220 for more details.

* All protected methods from CoreAdminHandler other than handleCustomAction() is

removed by SOLR-8476 and can
 no more be overridden. If you still wish to override those methods, override the
handleRequestBody()

* The PERSIST CoreAdmin action which was a NOOP and returned a deprecated message
has been removed. See SOLR-8476
for more details. The corresponding SolrJ action has also been removed.

* bin/post now defaults application/json files to the /update/json/docs end-point.

Use `-format solr` to force
files to the /update end-point. See SOLR-7042 for more details.

* In solrconfig.xml the <mergePolicy> element is deprecated in favor of a similar

<mergePolicyFactory> element,
the <mergeFactor> and <maxMergeDocs> elements are also deprecated, please see
SOLR-8621 for full details.

To migrate your existing solrconfig.xml, you can replace elements as follows:

<!-- TieredMergePolicy example -->

<!-- deprecated -->
<mergeFactor>??</mergeFactor>
<mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
<bool name="useCompoundFile">???</bool> <!-- deprecated since Lucene/Solr 4.4.0
-->
...
</mergePolicy>
<!-- replacement -->
<useCompoundFile>???</useCompoundFile> <!-- since Lucene/Solr 4.4.0 -->
<mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory">
<int name="maxMergeAtOnce">??</int> <!-- formerly the <mergeFactor> element
implicitly set this -->
<int name="segmentsPerTier">??</int> <!-- formerly the <mergeFactor> element
implicitly set this -->
...
</mergePolicyFactory>

<!-- Log(ByteSize|Doc)MergePolicy example -->

<!-- deprecated -->
<maxMergeDocs>????</maxMergeDocs>
<mergeFactor>??</mergeFactor>
<mergePolicy class="org.apache.lucene.index.Log?MergePolicy">
<bool name="useCompoundFile">???</bool> <!-- deprecated since Lucene/Solr 4.4.0
-->
...
</mergePolicy>
<!-- replacement -->
<useCompoundFile>???</useCompoundFile> <!-- since Lucene/Solr 4.4.0 -->
<mergePolicyFactory class="org.apache.solr.index.Log?MergePolicyFactory">
<int name="maxMergeDocs">????</int> <!-- formerly the <maxMergeDocs> element
set this -->
<int name="mergeFactor">??</int> <!-- formerly the <mergeFactor> element set
this -->
...
</mergePolicyFactory>

* Clearing up stored async collection api responses via REQUESTSTATUS call is now
deprecated and would be
removed in 6.0. See SOLR-8648 for more details.
* SOLR-6594: Deprecated the old schema API which will be removed in a later major
release

Detailed Change List

----------------------

New Features
----------------------

* SOLR-7928: Improve CheckIndex to work against HdfsDirectory

(Mike Drob, Gregory Chanan)

* SOLR-8378: Add upconfig and downconfig commands to the bin/solr script

(Erick Erickson)

* SOLR-8434: Add wildcard support to role, to match any role in

RuleBasedAuthorizationPlugin (noble)

* SOLR-4280: Allow specifying "spellcheck.maxResultsForSuggest" as a percentage of

filter
query results (Markus Jelsma via James Dyer)

* SOLR-8429: Add a flag 'blockUnknown' to BasicAuthPlugin to block unauthenticated

requests (noble)

* SOLR-8230: JSON Facet API: add "facet-info" into debug section of response when
debugQuery=true
(Michael Sun, yonik)

* SOLR-8428: RuleBasedAuthorizationPlugin adds an 'all' permission (noble)

* SOLR-5743: BlockJoinFacetComponent and BlockJoinDocSetFacetComponent for

calculating facets by
child.facet.field parameter with {!parent ..}.. query. They count facets on
children documents
aggregating (deduplicating) counts by parent documents (Dr. Oleg Savrasov via
Mikhail Khludnev)

* SOLR-8220: Read field from DocValues for non stored fields.

(Keith Laban, yonik, Erick Erickson, Ishan Chattopadhyaya, shalin)

* SOLR-8470: Make TTL of PKIAuthenticationPlugin's tokens configurable through a

system property
(pkiauth.ttl) (noble)

* SOLR-8477: Let users choose compression mode in SchemaCodecFactory (Tomás

Fernández Löbbe)

* SOLR-839: XML QueryParser support (defType=xmlparser)

Lucene includes a queryparser that supports the creation of Lucene queries from
XML.
The queries supported by lucene.queryparser.xml.CoreParser are now supported by
the newly
created solr.search.SolrCoreParser and in future SolrCoreParser could support
additional
queries also.
Example: <BooleanQuery fieldName="description">
<Clause occurs="must"> <TermQuery>shirt</TermQuery> </Clause>
<Clause occurs="mustnot"> <TermQuery>plain</TermQuery> </Clause>
 <Clause occurs="should"> <TermQuery>cotton</TermQuery> </Clause>
<Clause occurs="must">
<BooleanQuery fieldName="size">
<Clause occurs="should"> <TermsQuery>S M L</TermsQuery> </Clause>
</BooleanQuery>
</Clause>
</BooleanQuery>
(Erik Hatcher, Karl Wettin, Daniel Collins, Nathan Visagan, Ahmet Arslan,
Christine Poerschke)

* SOLR-8312: Add domain size and numBuckets to facet telemetry info (facet debug
info
for the new Facet Module). (Michael Sun, yonik)

* SOLR-8534: Add generic support for collection APIs to be async. Thus more actions
benefit from having async
support. The commands that additionally get async support are: delete/reload
collection, create/delete alias,
create/delete shard, delete replica, add/delete replica property, add/remove
role,
overseer status, balance shard unique, rebalance leaders, modify collection,
migrate state format (Varun Thacker)

* SOLR-4619: Improve PreAnalyzedField query analysis. (Andrzej Bialecki, Steve

Rowe)

* SOLR-8560: Added RequestStatusState enum which can be used when comparing states
of
asynchronous requests. (Shai Erera)

* SOLR-8586: added index fingerprint, a hash over all versions currently in the
index.
PeerSync now uses this to check if replicas are in sync. (yonik)

* SOLR-8500: Allow the number of threads ConcurrentUpdateSolrClient

StreamingSolrClients configurable by a
system property. NOTE: this is an expert option and can result in more often
needing to do full index replication
for recovery, the sweet spot for using this is very high volume, leader-only
indexing. (Tim Potter, Erick Erickson)

* SOLR-8642: SOLR allows creation of collections with invalid names

(Jason Gerlowski via Erick Erickson)

* SOLR-8621: Deprecate <mergePolicy> in favor of <mergePolicyFactory>. It allows to

configure
both the "simple" merge policies, but also more advanced ones, e.g.
UpgradeIndexMergePolicy.
(Christine Poerschke, Shai Erera)

* SOLR-8648: DELETESTATUS API for selective deletion and flushing of stored async
collection API responses.
(Anshum Gupta)

* SOLR-8466: adding facet.method=uif to bring back UnInvertedField faceting which

is used to work on
facet.method=fc. It's more performant for rarely changing indexes. Note: it
ignores prefix and contains yet.
(Jamie Johnson via Mikhail Khludnev)
Bug Fixes
----------------------

* SOLR-8175: Word Break Spellchecker would throw AIOOBE with certain queries
containing
"should" clauses. (Ryan Josal via James Dyer)

* SOLR-2556: The default spellcheck query converter was ignoring terms consisting
entirely
of digits. (James Dyer)

* SOLR-8366: ConcurrentUpdateSolrClient attempts to use response's content type as

charset encoding
for parsing exception. (shalin)

* SOLR-6271: Fix ConjunctionSolrSpellChecker to not compare StringDistance by

instance.
(Igor Kostromin via James Dyer)

* SOLR-7304: Fix Spellcheck Collate to not invalidate range queries. (James Dyer)

* SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to

fetch TGT for every request (Ishan Chattopadhyaya via noble)

* SOLR-8367: Fix the LeaderInitiatedRecovery 'all replicas participate' fail-safe.

(Mark Miller, Mike Drob)

* SOLR-8401: Windows start script fails when executed from a different drive.
(Nicolas Gavalda via Erick Erickson)

* SOLR-6992: Fix "Files" UI to show the managed-schema file as well.

(Shawn Heisey, Varun Thacker)

* SOLR-2649: MM ignored in edismax queries with operators.

(Greg Pendlebury, Jan Høydahl et. al. via Erick Erickson)

* SOLR-8372: Canceled recovery can rarely lead to inconsistent shards:

If a replica is recovering via index replication, and that recovery fails
(for example if the leader goes down), and then some more updates are received
(there could be a few left to be processed from the leader that just went down),
and then that replica is brought down, it will think it is up-to-date when
restarted. (shalin, Mark Miller, yonik)

* SOLR-8419: TermVectorComponent for distributed search when distrib.singlePass

could include term
vectors for documents that matched the query yet weren't in the returned
documents. (David Smiley)

* SOLR-8015: HdfsLock may fail to close a FileSystem instance if it cannot

immediately
obtain an index lock. (Mark Miller)

* SOLR-8422: When authentication enabled, requests fail if sent to a node that

doesn't host
the collection (noble)

* SOLR-8059: &debug=results for distributed search when distrib.singlePass

(sometimes activated
 automatically) could result in an NPE. (David Smiley, Markus Jelsma)

* SOLR-8460: /analysis/field could throw exceptions for custom attributes. (David

Smiley, Uwe Schindler)

* SOLR-8276: Atomic updates and realtime-get do not work with non-stored docvalues.
(Ishan Chattopadhyaya, yonik via shalin)

* SOLR-7462: AIOOBE in RecordingJSONParser (Scott Dawson, noble)

* SOLR-8494: SimplePostTool and therefore the bin/post script cannot upload files
larger than 2.1GB. (shalin)

* SOLR-8451: We should not call method.abort in HttpSolrClient or

HttpSolrCall#remoteQuery and
HttpSolrCall#remoteQuery should not close streams. (Mark Miller)

* SOLR-8450: Our HttpClient retry policy is too permissive. (Mark Miller, shalin)

* SOLR-8533: Raise default maxUpdateConnections and maxUpdateConnectionsPerHost to

100k each.
(Mark Miller)

* SOLR-8453: Solr should attempt to consume the request inputstream on errors as we

cannot
count on the container to do it. (Mark Miller, Greg Wilkins, yonik, Joakim
Erdfelt)

* SOLR-6279: cores?action=UNLOAD now waits for the core to close before

unregistering it from ZK.
(Christine Poerschke)

* SOLR-2798: Fixed local params to work correctly with multivalued params

(Demian Katz via hossman)

* SOLR-8541: Highlighting a geo RPT field would throw an NPE instead of doing
nothing.
(Pawel Rog via David Smiley)

* SOLR-8548: Core discovery was not following symlinks (Aaron LaBella via Alan
Woodward)

* SOLR-8564: Fix Embedded ZooKeeper to use <solrHome>/zoo_data for it's data

directory

* SOLR-8371: Try and prevent too many recovery requests from stacking up and clean
up some faulty
cancel recovery logic. (Mark Miller)

* SOLR-8582 : memory leak in JsonRecordReader affecting /update/json/docs. Large

payloads
cause OOM (noble, shalin)

* SOLR-8605: Regular expression queries starting with escaped forward slash caused
an exception. (Scott Blum, yonik)

* SOLR-8607: The Schema API refuses to add new fields that match existing dynamic
fields.
(Jan Høydahl, Steve Rowe)
* SOLR-8575: Fix HDFSLogReader replay status numbers, a performance bug where we
can reopen
FSDataInputStream much too often, and an hdfs tlog data integrity bug.
(Mark Miller, Patrick Dvorack, yonik)

* SOLR-8651: The commitWithin parameter is not passed on for deleteById in

UpdateRequest in
distributed queries (Jessica Cheng Mallet via Erick Erickson)

* SOLR-8551: Make collection deletion more robust. (Mark Miller)

Optimizations
----------------------

* SOLR-8501: Specify the entity request size when known in HttpSolrClient. (Mark
Miller)

* SOLR-8615: Just like creating cores, we should use multiple threads when closing
cores.
(Mark Miller)

* SOLR-7281: Add an overseer action to publish an entire node as 'down'. (Mark

Miller, shalin)

* SOLR-8669: Non binary responses use chunked encoding because we flush the
outputstream early.
(Mark Miller)

Other Changes
----------------------

* LUCENE-6900: Added test for score ordered grouping, and refactored

TopGroupsResultTransformer.
(David Smiley)

* SOLR-8336: CoreDescriptor now takes a Path for its instance directory, rather
than a String (Alan Woodward)

* SOLR-8351: Improve HdfsDirectory toString representation

(Mike Drob via Gregory Chanan)

* SOLR-8321: add a (SolrQueryRequest free) SortSpecParsing.parseSortSpec variant

(Christine Poerschke)

* SOLR-8338: in OverseerTest replace strings such as "collection1" and "state" with

variable
or enum equivalent (Christine Poerschke)

* SOLR-8333: Several API tweaks so that public APIs were no longer refering to
private classes
(ehatcher, Shawn Heisey, hossman)

* SOLR-8357: UpdateLog.RecentUpdates now implements Closeable (Alan Woodward)

* SOLR-8339: Refactor SolrDocument and SolrInputDocument to have a common base

abstract class
called SolrDocumentBase. Deprecated methods toSolrInputDocument and
toSolrDocument in ClientUtils.
 (Ishan Chattopadhyaya via shalin)

* SOLR-8353: Support regex for skipping license checksums (Gregory Chanan)

* SOLR-8313: SimpleQueryParser doesn't use MultiTermAnalysis for Fuzzy Queries (Tom

Hill via Erick Erickson)

* SOLR-8359: Restrict child classes from using parent logger's state

(Jason Gerlowski, Mike Drob, Anshum Gupta)

* SOLR-8131: All example config sets now explicitly use the

ManagedIndexSchemaFactory
instead of ClassicIndexSchemaFactory. This means that the Schema APIs (
/<collection>/schema )
are enabled by default and the schema is mutable. The schema file will be called
managed-schema
(Uwe Schindler, shalin, Varun Thacker)

* SOLR-8381: Cleanup data_driven managed-schema and solrconfig.xml files. Commented

out copyFields are removed
and solrconfig.xml doesn't refer to field which are not defined. (Varun Thacker)

* SOLR-7774: revise BasicDistributedZkTest.test logic w.r.t. 'commitWithin did not

work on some nodes'
(Christine Poerschke)

* SOLR-8360: simplify ExternalFileField.getValueSource implementation (Christine

Poerschke)

* SOLR-8387: All example configs shipped with Solr explicitly use

ManagedIndexSchemaFactory, the schema file will
be called managed-schema instead of schema.xml . It is not advised to use hand
edit the managed-schema. You should
use the schema APIs instead ( /<collection>/schema ) . If you do not want this
behaviour in the example configs,
before you start solr rename managed-schema to schema.xml and change the
schemaFactory in solrconfig.xml file
to explicitly use ClassicIndexSchemaFactory instead : <schemaFactory
class="ClassicIndexSchemaFactory"/>
(Varun Thacker)

* SOLR-8305: replace LatLonType.getValueSource's QParser use (Christine Poerschke)

* SOLR-8388: factor out response/TestSolrQueryResponse.java from

servlet/ResponseHeaderTest.java
more TestSolrQueryResponse.java tests; add SolrReturnFields.toString method,
ReturnFieldsTest.testToString test;
(Christine Poerschke)

* SOLR-8383: SolrCore.java + QParserPlugin.java container initialCapacity tweaks

(Christine Poerschke, Mike Drob)

* LUCENE-6925: add RandomForceMergePolicy class in test-framework (Christine

Poerschke)

* SOLR-8404: tweak SolrQueryResponse.getToLogAsString, add

TestSolrQueryResponse.testToLog (Christine Poerschke)

* SOLR-8352: randomise unload order in

UnloadDistributedZkTest.testUnloadShardAndCollection (Christine Poerschke)

* SOLR-8414: AbstractDistribZkTestBase.verifyReplicaStatus could throw NPE

(Christine Poerschke)

* SOLR-8410: Add all read paths to 'read' permission in

RuleBasedAuthorizationPlugin (noble)

* SOLR-8279: Add a new test fault injection approach and a new SolrCloud test that
stops and starts the cluster
while indexing data and with random faults. (Mark Miller)

* SOLR-8419: TermVectorComponent for distributed search now requires a uniqueKey in

the schema. Also, it no longer
returns "uniqueKeyField" in the response. (David Smiley)

* SOLR-8317: add & use responseHeader and response accessors to SolrQueryResponse.

(Christine Poerschke)

* SOLR-8452: replace "partialResults" occurrences with

SolrQueryResponse.RESPONSE_HEADER_PARTIAL_RESULTS_KEY
(Christine Poerschke)

* SOLR-8454: ZkStateReader logging improvements and cleanup of dead code (Shai

Erera, Anshum Gupta)

* SOLR-8455: RecovertStrategy logging improvements and sleep-between-recovery-

attempts bug fix.
(Shai Erera)

* SOLR-8481: TestSearchPerf no longer needs to duplicate SolrIndexSearcher.

(NO_CHECK_QCACHE|NO_CHECK_FILTERCACHE)
(Christine Poerschke)

* SOLR-8486: No longer require jar/unzip for bin/solr (Steven E. Harris, janhoy)

* SOLR-8483: relocate 'IMPORTANT NOTE' in open-exchange-rates.json test-file to

avoid
OpenExchangeRatesOrgProvider.java warnings (Christine Poerschke)

* SOLR-8489: TestMiniSolrCloudCluster.createCollection to support extra &

alternative collectionProperties
(Christine Poerschke)

* SOLR-8482: add & use QueryCommand.[gs]etTerminateEarly accessors. (Christine

Poerschke)

* SOLR-8498: Improve error message when a large value is stored in an indexed

string field. (shalin)

* SOLR-8484: refactor update/SolrIndexConfig.LOCK_TYPE_* into

core/DirectoryFactory.LOCK_TYPE_*
(Christine Poerschke)

* SOLR-8504: (IndexSchema|SolrIndexConfig)Test: private static finals for

solrconfig.xml and schema.xml String literals. (Christine Poerschke)

* SOLR-8505: core/DirectoryFactory.LOCK_TYPE_HDFS - add & use it instead of String

literals
 (Christine Poerschke)

* SOLR-7042: bin/post now uses /update/json/docs for application/json content

types, including support for
.jsonl (JSON Lines) files. (Erik Hatcher and shalin)

* SOLR-8476: Refactor and cleanup CoreAdminHandler (noble, Varun Thacker)

* SOLR-8535: Support forcing define-lucene-javadoc-url to be local (Gregory Chanan)

* SOLR-8549: Solr start script checks for cores which have failed to load as well
before attempting to
create a core with the same name (Varun Thacker)

* SOLR-8555: SearchGroupShardResponseProcessor (initialCapacity) tweaks (Christine

Poerschke)

* LUCENE-6978: Refactor several code places that lookup locales

by string name to use BCP47 locale tag instead. LuceneTestCase
now also prints locales on failing tests this way. In addition,
several places in Solr now additionally support BCP47 in config
files. (Uwe Schindler, Robert Muir)

* SOLR-7907: Remove CLUSTERSTATUS related exclusivity checks while running commands

in the Overseer because the
CLUSTERSTATUS request is served by the individual nodes itself and not via the
Overseer node (Varun Thacker)

* SOLR-8566: various initialCapacity tweaks (Fix Versions: trunk 5.5)

(Christine Poerschke)

* SOLR-8565: add & use CommonParams.(ROWS|START)_DEFAULT constants (Christine

Poerschke)

* SOLR-8595: Use BinaryRequestWriter by default in HttpSolrClient and

ConcurrentUpdateSolrClient. (shalin)

* SOLR-8597: add default, no-op QParserPlugin.init(NamedList) method (Christine

Poerschke)

* SOLR-7968: Make QueryComponent more extensible. (Markus Jelsma via David Smiley)

* SOLR-8600: add & use ReRankQParserPlugin parameter [default] constants,

changed ReRankQuery.toString to use StringBuilder. (Christine Poerschke)

* SOLR-8308: Core gets inaccessible after RENAME operation with special characters
(Erik Hatcher, Erick Erickson)

* SOLR-3141: Warn in logs when expensive optimize calls are made (yonik, janhoy)

================== 5.4.1 ==================

Bug Fixes
----------------------

* SOLR-8460: /analysis/field could throw exceptions for custom attributes. (David

Smiley, Uwe Schindler)

* SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to

 fetch TGT for every request (Ishan Chattopadhyaya via noble)

* SOLR-8059: &debug=results for distributed search when distrib.singlePass

(sometimes activated
automatically) could result in an NPE. (David Smiley, Markus Jelsma)

* SOLR-8422: When authentication enabled, requests fail if sent to a node that

doesn't host
the collection (noble)

* SOLR-7462: AIOOBE in RecordingJSONParser (Scott Dawson, noble)

* SOLR-8496: SolrIndexSearcher.getDocSet(List<Query>) incorrectly included deleted

documents
when all of the queries were uncached (or there was no filter cache). This
caused
multi-select faceting (including the JSON Facet API) to include deleted doc
counts
when the remaining non-excluded filters were all uncached. This bug was first
introduced in 5.3.0
(Andreas Müller, Vasiliy Bout, Erick Erickson, Shawn Heisey, Hossman, yonik)

* SOLR-8418: Adapt to changes in LUCENE-6590 for use of boosts with MLTHandler and
Simple/CloudMLTQParser (Jens Wille, Ramkumar Aiyengar)

New Features
----------------------

* SOLR-8470: Make TTL of PKIAuthenticationPlugin's tokens configurable through a

system property
(pkiauth.ttl) (noble)

================== 5.4.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.10.4
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.2.13.v20150730

Upgrading from Solr 5.3

-----------------------

* DefaultSimilarityFactory has been renamed to ClassicSimilarityFactory to match

the underlying rename of
DefaultSimilarity to ClassicSimilarity and the (eventual) move away from using it
as a default.
If you currently have DefaultSimilarityFactory explicitly referenced in your
schema.xml, you will now get
a warning urging you to edit your config to use the functionally identical
ClassicSimilarityFactory.
DefaultSimilarityFactory will be removed completely in Solr 6. See SOLR-8239 for
more details.
* SOLR-7859: The following APIs are now deprecated:
- SolrCore.getStartTime: Use SolrCore.getStartTimeStamp instead.
- SolrIndexSearcher.getOpenTime: Use SolrIndexSearcher.getOpenTimeStamp instead.

* SOLR-8307: EmptyEntityResolver was moved from core to solrj, and moved from the
org.apache.solr.util
package to org.apache.solr.common. If you are using this class, you will need to
adjust the import package.

* Logger declarations in most source files have changed to code that

no longer needs to explicitly state the class name. This fixes situations
where a logger for a different class was incorrectly used. See SOLR-8324
and its sub-issues for details.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-5756: A utility Collection API to move a collection from shared

clusterstate.json (stateFormat=1,
default until 4.x) to the per-collection state.json stored in ZooKeeper
(stateFormat=2,
default since 5.0) seamlessly without any application down-time.
Example:
http://localhost:8983/solr/admin/collections?
action=MIGRATESTATEFORMAT&collection=<collection_name>
(Noble Paul, Scott Blum, shalin)

* SOLR-7219: filterCache access added to the solr query syntax.

Example: description:HDTV OR filter(+promotion:tv +promotion_date:[NOW/DAY TO
NOW/DAY+7DAY])
(yonik)

* SOLR-7775: Allow fromIndex parameter to ScoreJoinQParserPlugin {!join score=..

fromIndex=..}..
to refer to a single-sharded collection that has a replica on all nodes where
there is a
replica in the to index (Andrei Beliakov via Mikhail Khludnev)

* SOLR-7961: Print Solr's version with command bin/solr version (janhoy)

* SOLR-7789: Introduce a ConfigSet management API (Gregory Chanan)

* SOLR-4316: Add a collections dropdown to angular admin UI (Upayavira, Shalin

Shekhar Mangar)

* SOLR-7915: Provide pluggable context tool support for VelocityResponseWriter

(Erik Hatcher)

* LUCENE-6795: SystemInfoHandler was improved to also show detailed operating

system statistics on IBM J9 virtual machines. It also no longer fails on Java 9
with Jigsaw module system. (Uwe Schindler)

* SOLR-8053: Basic auth support in SolrJ (noble)

* SOLR-7995: Add a LIST command to ConfigSets API (Gregory Chanan)

* SOLR-4388: In Angular UI, add a Collections UI when in cloud mode (Upayavira)

* SOLR-7858, SOLR-8199: Add links between original and new Admin UIs (Upayavira)

* SOLR-7888: Analyzing suggesters can now filter suggestions by a context field

(Arcadius Ahouansou, janhoy)

* SOLR-8217: JSON Facet API: add "method" param to terms/field facets to give an
execution
hint for what method should be used to facet. (yonik)

* SOLR-8113: CloneFieldUpdateProcessorFactory now supports choosing a "dest" field

name based on a regex
pattern and replacement init options. (Gus Heck, hossman)

* SOLR-8139: Create/delete fields/dynamic fields/copy fields via schema tab on

Angular UI

* SOLR-8166: Introduce possibility to configure ParseContext in

ExtractingRequestHandler/ExtractingDocumentLoader (Andriy Binetsky
via Uwe Schindler)

* SOLR-7569: A collection API to force elect a leader, called FORCELEADER, when all
replicas in a shard are down
(Ishan Chattopadhyaya, Mark Miller, shalin, noble)

* SOLR-6168: Add a 'sort' local param to the collapse QParser to support using
complex sort options
to select the representitive doc for each collapsed group. (Umesh Prasad,
hossman)

* SOLR-8329: SchemaSimilarityFactory now supports a 'defaultSimFromFieldType' init

option for using
a fieldType name to identify which Similarity to use as a default. (hossman)

* SOLR-7912: Add boost support, and also exclude the queried document in
MoreLikeThis QParser
(Jens Wille via Anshum Gupta)

Bug Fixes
----------------------

* SOLR-7859: Fix usage of currentTimeMillis instead of nanoTime in multiple places,

whitelist valid uses of currentTimeMillis (Ramkumar Aiyengar)

* SOLR-7836: Possible deadlock when closing refcounted index writers.

(Jessica Cheng Mallet, Erick Erickson, Mark Miller, yonik)

* SOLR-7869: Overseer does not handle BadVersionException correctly and, in some

cases,
can go into an infinite loop if cluster state in ZooKeeper is modified
externally.
(Scott Blum, shalin)

* SOLR-7920: Resolve XSS issue in Admin UI Schema Browser (David Chiu via
Upayavira)

* SOLR-7935: Fix very rare race condition that can cause an update to fail
 via NullPointerException during a core reload. (yonik)

* SOLR-7941: multivalued params are concatenated when using config API (noble)

* SOLR-7956: There are interrupts on shutdown in places that can cause

ChannelAlreadyClosed
exceptions which prevents proper closing of transaction logs, interfere with the
IndexWriter,
the hdfs client and other things. (Mark Miller, Scott Blum)

* SOLR-7954: Fixed an integer overflow bug in the HyperLogLog code used by the
'cardinality' option
of stats.field to prevent ArrayIndexOutOfBoundsException in a distributed search
when a large precision
is selected and a large number of values exist in each shard (hossman)

* SOLR-7844: Zookeeper session expiry during shard leader election can cause
multiple leaders.
(Mike Roberts, Mark Miller, Jessica Cheng)

* SOLR-7984: wrong and misleading error message 'no default request handler is
registered' (noble, hossman)

* SOLR-8001: Fixed bugs in field(foo,min) and field(foo,max) when some docs have no
values
(David Smiley, hossman)

* SOLR-7819: ZK connection loss or session timeout do not stall indexing threads

anymore. All activity
related to leader initiated recovery is performed by a dedicated LIR thread in
the background.
(Ramkumar Aiyengar, shalin)

* SOLR-7746: Ping requests stopped working with distrib=true in Solr 5.2.1.

(Alexey Serba, Michael Sun via Gregory Chanan)

* SOLR-6547: ClassCastException in SolrResponseBase.getQTime on update response

from CloudSolrClient
when parallelUpdates is enabled (default) and multiple docs are sent as a single
update.
(kevin, hossman, shalin)

* SOLR-8058: Fix the exclusion filter so that collections that start with js, css,
img, tpl
can be accessed. (Upayavira, Steve Rowe, Anshum Gupta)

* SOLR-8069: Ensure that only the valid ZooKeeper registered leader can put a
replica into Leader
Initiated Recovery. (Mark Miller, Jessica Cheng, Anshum Gupta)

* SOLR-8077: Replication can still cause index corruption. (Mark Miller)

* SOLR-8104: Config API does not work for spellchecker (noble)

* SOLR-8095: Allow disabling HDFS Locality Metrics and disable by default as it may
have performance
implications on rapidly changing indexes. (Mike Drob via Mark Miller)

* SOLR-8085: Fix a variety of issues that can result in replicas getting out of
sync. (yonik, Mark Miller)

* SOLR-8094: HdfsUpdateLog should not replay buffered documents as a replacement to

dropping them.
(Mark Miller)

* SOLR-8075: Leader Initiated Recovery should not stop a leader that participated
in an election with all
of it's replicas from becoming a valid leader. (Mark Miller)

* SOLR-8072: Rebalance leaders feature does not set CloudDescriptor#isLeader to

false when bumping leaders.
(Mark Miller)

* SOLR-7666: Many small fixes to Angular UI (Upayavira, Alexandre Rafalovitch)

* SOLR-7967: AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet

is immutable (Gregory Chanan)

* SOLR-6188: Skip the automatic loading of resources in the "lib" subdirectory

by SolrResourceLoader, but only if we are loading resources from the solr
home directory. Fixes the inability to use ICU analysis components with a
"solr." prefix on the classname. (Shawn Heisey)

* SOLR-8130: Solr's hdfs safe mode detection does not catch all cases of being in
safe mode.
(Mark Miller, Mike Drob)

* SOLR-8128: Set v.locale specified locale for all LocaleConfig extending

VelocityResponseWriter tools.
(Erik Hatcher)

* SOLR-8152: Overseer Task Processor/Queue can miss responses, leading to timeouts.

(Gregory Chanan)

* SOLR-8107: bin/solr -f should use exec to start the JVM (Martijn Koster via
Timothy Potter)

* SOLR-8050: Partial update on document with multivalued date field fails to parse
date and can
also fail to remove dates in some cases. (Burkhard Buelte, Luc Vanlerberghe,
shalin)

* SOLR-8167: Authorization framework does not work with POST params (noble)

* SOLR-8162: JmxMonitoredMap#clear triggers a query on all the MBeans thus

generating lots of warnings.
(Marius Dumitru Florea, shalin)

* SOLR-7843: DataImportHandler's delta imports leak memory because the delta keys
are kept in memory
and not cleared after the process is finished. (Pablo Lozano via shalin)

* SOLR-8189: eTag calculation during HTTP Cache Validation uses unsynchronized

WeakHashMap causing
threads to be stuck in runnable state. (shalin)

* SOLR-7993: Raw json output for fields stopped working in 5.3.0 when requested
fields do not include
 the unique key field name. (Bill Bell, Ryan McKinley via shalin)

* SOLR-8192: JSON Facet API allBuckets:true did not work correctly when faceting
on a multi-valued field with sub-facets / facet functions. (yonik)

* SOLR-8206: JSON Facet API limit:0 did not always work correctly. (yonik)

* SOLR-8126: update-<component-name> does not work if the component is only

present in solrconfig.xml (noble)

* SOLR-8203: Stop processing updates more quickly on node shutdown. When a node
is shut down, streaming updates would continue, but new update requests would
be aborted. This can cause big update reorders that can cause replicas to
get out of sync. (Mark Miller, yonik)

* SOLR-6406: ConcurrentUpdateSolrClient hang in blockUntilFinished. If updates are

still
flowing and shutdown is called on the executor service used by
ConcurrentUpdateSolrClient,
a race condition can cause that client to hang in blockUntilFinished.
(Mark Miller, yonik)

* SOLR-8215: Only active replicas should handle incoming requests against a

collection (Varun Thacker)

* SOLR-8223: Avoid accidentally swallowing OutOfMemoryError (in

LeaderInitiatedRecoveryThread.java
or CoreContainer.java) (Mike Drob via Christine Poerschke)

* SOLR-8255: MiniSolrCloudCluster needs to use a thread-safe list to keep track

of its child nodes (Alan Woodward)

* SOLR-8254: HttpSolrCore.getCoreByCollection() can throw NPE (Alan Woodward,

Mark Miller)

* SOLR-8262: Comment out /stream handler from sample solrconfig.xml's for security
reasons
(Joel Bernstein)

* SOLR-7989: After a new leader is elected it should change it's state to ACTIVE
even
if the last published state is something else if it has already registered with
ZK.
(Ishan Chattopadhyaya, Mark Miller via noble)

* SOLR-8287: TrieDoubleField and TrieLongField now override toNativeType

(Ishan Chattopadhyaya via Christine Poerschke)

* SOLR-8284: JSON Facet API - fix NPEs when short form "sort:index" or "sort:count"
are used. (Michael Sun via yonik)

* SOLR-8295: Fix NPE in collapse QParser when collapse field is missing from all
docs in a segment (hossman)

* SOLR-8280: Fixed bug in SimilarityFactory initialization that prevented

SolrCoreAware factories -- such
as SchemaSimilarityFactory -- from functioning properly with managed schema
features. (hossman)
* SOLR-5971: Fix error 'Illegal character in query' when proxying request.
(Uwe Schindler, Ishan Chattopadhyaya, Eric Bus)

* SOLR-8307: Fix XXE vulnerability in MBeansHandler "diff" feature (Erik Hatcher)

* SOLR-8073: Solr fails to start on Windows with obscure errors when using relative
path.
(Alexandre Rafalovitch, Ishan Chattopadhyaya via shalin)

* SOLR-7169: bin/solr status should return exit code 3, not 0 if Solr is not
running
(Dominik Siebel via Timothy Potter)

* SOLR-8341: Fix JSON Facet API excludeTags when specified in the

form of domain:{excludeTags:mytag} (yonik)

* SOLR-8326: If BasicAuth enabled, inter node requests fail after node restart
(noble, Anshum Gupta)

* SOLR-8340: Fixed NullPointerException in HighlightComponent.

(zengjie via Christine Poerschke)

* SOLR-8355: update permissions were failing node recovery (noble , Anshum Gupta)

Optimizations
----------------------

* SOLR-7918: Filter (DocSet) production from term queries has been optimized and
is anywhere from 20% to over 100% faster and produces less garbage on average.
(yonik)

* SOLR-6760: New optimized DistributedQueue implementation for overseer increases

message processing performance by ~470%.
(Noble Paul, Scott Blum, shalin)

* SOLR-6629: Watch /collections zk node on all nodes so that cluster state updates
are more efficient especially when cluster has a mix of collections in
stateFormat=1
and stateFormat=2. (Scott Blum, shalin)

* SOLR-7971: Reduce memory allocated by JavaBinCodec to encode small strings by an

amount
equal to the string.length(). JavaBinCodec now uses a double pass approach to
write strings
larger than 64KB to avoid allocating buffer memory equal to string's UTF8 size.
(yonik, Steve Rowe, Mikhail Khludnev, Noble Paul, shalin)

* SOLR-7983: Utils.toUTF8 uses larger buffer than necessary for holding UTF8 data.
(shalin)

* SOLR-8222: JSON Facet API optimization to faceting by count on docvalue fields

(or indexed fields
with method=dv) when there are multiple hits expected for enoug buckets. For
example, this
more than doubled the performance of faceting 5M documents over a field with 1M
unique values.
(yonik)
* SOLR-8288: DistributedUpdateProcessor#doFinish should explicitly check and ensure
it
does not try to put itself into LIR. (Mark Miller)

Other Changes
----------------------

* SOLR-8294: Cleanup solrconfig.xmls under solr/example/example-DIH/solr (removed

obsolete clustering handler sections). (Dawid Weiss)

* SOLR-7969: Unavailable clustering engines should not fail the core. (Dawid Weiss)

* SOLR-7790, SOLR-7791: Update Carrot2 clustering component to

version 3.10.4. Upgrade HPPC library to version 0.7.1. (Dawid Weiss)

* SOLR-7831: Start Scripts: Allow a configurable stack size [-Xss] (Steve Davids
via Mark Miller)

* SOLR-7870: Write a test which asserts that requests to stateFormat=2 collection

succeed on a node
even after all local replicas of that collection have been removed.
(Scott Blum via shalin)

* SOLR-7902: Split out use of child timers from RTimer to a sub-class

(Ramkumar Aiyengar)

* SOLR-7943: Upgrade Jetty to 9.2.13.v20150730. (Bill Bell, shalin)

* SOLR-7007: DistributedUpdateProcessor now logs replay flag as boolean instead of

int
(Mike Drob via Christine Poerschke)

* SOLR-7960: Start scripts now gives generic help for bin/solr -h and bin/solr --
help (janhoy)

* SOLR-7970: Factor out a SearchGroupsFieldCommandResult class.

(Christine Poerschke)

* SOLR-7942: Previously removed unlockOnStartup option (LUCENE-6508) now logs

warning if configured,
will be an error in 6.0. Also improved error msg if an index is locked on
startup (hossman)

* SOLR-7979: Fix two typos (in a CoreAdminHandler log message and a

TestCloudPivotFacet comment).
(Mike Drob via Christine Poerschke)

* SOLR-7966: Solr Admin UI Solr now sets the HTTP header X-Frame-Options to DENY
to avoid clickjacking. (yonik)

* SOLR-7999: SolrRequestParser tests no longer depend on external URLs

that may fail to work. (Uwe Schindler)

* SOLR-8034: Leader no longer puts replicas in recovery in case of a failed update,

when minRF
isn't achieved. (Jessica Cheng, Timothy Potter, Anshum Gupta)

* SOLR-8066: SolrCore.checkStale method doesn't restore interrupt status. (shalin)

* SOLR-8068: Throw a SolrException if the core container has initialization errors
or is
shutting down (Ishan Chattopadhyaya, Noble Paul, Anshum Gupta)

* SOLR-8083: Convert the ZookeeperInfoServlet to a handler at /admin/zookeeper

(noble)

* SOLR-8025: remove unnecessary ResponseBuilder.getQueryCommand() calls (Christine

Poerschke)

* SOLR-8150: Fix build failure due to too much output from QueryResponseTest
(janhoy)

* SOLR-8151: OverseerCollectionMessageHandler was logging info data as WARN

(Alan Woodward)

* SOLR-8116: SearchGroupsResultTransformer tweaks (String literals, list/map

initialCapacity)
(Christine Poerschke)

* SOLR-8114: in Grouping.java rename groupSort and sort to withinGroupSort and

groupSort
(Christine Poerschke)

* SOLR-8074: LoadAdminUIServlet directly references admin.html (Mark Miller,

Upayavira)

* SOLR-8195: IndexFetcher download trace now includes bytes-downloaded[-per-second]

(Christine Poerschke)

* SOLR-4854: Add a test to assert that [elevated] DocTransfer works correctly with
javabin
response format. (Ray, shalin)

* SOLR-8196: TestMiniSolrCloudCluster.testStopAllStartAll case plus necessary

MiniSolrCloudCluster tweak (Christine Poerschke)

* SOLR-8221: MiniSolrCloudCluster should create subdirectories for its nodes

(Alan Woodward)

* SOLR-8218: DistributedUpdateProcessor (initialCapacity) tweaks

(Christine Poerschke)

* SOLR-8147: contrib/analytics FieldFacetAccumulator now throws IOException instead

of SolrException
(Scott Stults via Christine Poerschke)

* SOLR-8239: Added ClassicSimilarityFactory, marked DefaultSimilarityFactory as

deprecated. (hossman)

* SOLR-8253: AbstractDistribZkTestBase can sometimes fail to shut down its

ZKServer (Alan Woodward)

* SOLR-8260: Use NIO2 APIs in core discovery (Alan Woodward)

* SOLR-8259: Deprecate JettySolrRunner.getDispatchFilter(), add

.getSolrDispatchFilter() and .getCoreContainer() (Alan Woodward)

* SOLR-8278: Use NIO2 APIs in ConfigSetService (Alan Woodward)

* SOLR-8286: Remove instances of solr.hdfs.blockcache.write.enabled from tests
and docs (Gregory Chanan)

* SOLR-8269: Upgrade commons-collections to 3.2.2. This fixes a known serialization

vulnerability (janhoy)

* SOLR-8246: Fix SolrCLI to clean the config directory in case creating a core
failed.
(Jason Gerlowski via Shai Erera)

* SOLR-8290: remove SchemaField.checkFieldCacheSource's unused QParser argument

(Christine Poerschke)

* SOLR-8300: Use constants for the /overseer_elect znode (Varun Thacker)

* SOLR-8283: factor out StrParser from QueryParsing.StrParser and

SortSpecParsing[Test]
from QueryParsing[Test] (Christine Poerschke)

* SOLR-8298: small preferLocalShards implementation refactor (Christine Poerschke)

* SOLR-8315: Removed default core checks in the dispatch filter since we don't have
a default
core anymore (Varun Thacker)

* SOLR-8302: SolrResourceLoader now takes a Path as its instance directory (Alan

Woodward, Shawn Heisey)

* SOLR-8303: CustomBufferedIndexInput now includes resource description when

throwing EOFException. (Mike Drob via Uwe Schindler)

* SOLR-8194: Improve error reporting for null documents in UpdateRequest (Markus

Jelsma, Alan Woodward)

* SOLR-8277: (Search|Top)GroupsFieldCommand tweaks (Christine Poerschke)

* SOLR-8299: ConfigSet DELETE operation no longer allows deletion of config sets

that
are currently in use by other collections (Anshum Gupta)

* SOLR-8101: Improve Linux service installation script (Sergey Urushkin via Timothy
Potter)

* SOLR-8180: jcl-over-slf4j should have officially been a SolrJ dependency; it now

is.
(David Smiley, Kevin Risden)

* SOLR-8330: Standardize and fix logger creation and usage so that they aren't
shared
across source files.(Jason Gerlowski, Uwe Schindler, Anshum Gupta)

* SOLR-8363: Fix check-example-lucene-match-version Ant task and addVersion.py

script to
check and update luceneMatchVersion under solr/example/ configs as well logic.
(Varun Thacker)

================== 5.3.2 ==================

Bug Fixes
----------------------

* SOLR-8460: /analysis/field could throw exceptions for custom attributes. (David

Smiley, Uwe Schindler)

* SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to

fetch TGT for every request (Ishan Chattopadhyaya via noble)

* SOLR-8340: Fixed NullPointerException in HighlightComponent. (zengjie via

Christine Poerschke)

* SOLR-8059: &debug=results for distributed search when distrib.singlePass

(sometimes activated
automatically) could result in an NPE. (David Smiley, Markus Jelsma)

* SOLR-8167: Authorization framework does not work with POST params (noble)

* SOLR-8355: update permissions were failing node recovery (noble , Anshum Gupta)

* SOLR-8326: If BasicAuth enabled, inter node requests fail after node restart
(noble, Anshum Gupta)

* SOLR-8269: Upgrade commons-collections to 3.2.2. This fixes a known serialization

vulnerability (janhoy)

* SOLR-8422: When authentication enabled, requests fail if sent to a node that

doesn't host
the collection (noble)

* SOLR-8496: SolrIndexSearcher.getDocSet(List<Query>) incorrectly included deleted

documents
when all of the queries were uncached (or there was no filter cache). This
caused
multi-select faceting (including the JSON Facet API) to include deleted doc
counts
when the remaining non-excluded filters were all uncached. This bug was first
introduced in 5.3.0
(Andreas Müller, Vasiliy Bout, Erick Erickson, Shawn Heisey, Hossman, yonik)

================== 5.3.1 ==================

Bug Fixes
----------------------

* SOLR-7949: Resolve XSS issue in Admin UI stats page (David Chiu via janhoy)

* SOLR-8000: security.json is not loaded on server start (noble)

* SOLR-8004: RuleBasedAuthorization plugin does not work for the collection-admin-

edit permission (noble)

* SOLR-7972: Fix VelocityResponseWriter template encoding issue.

Templates must be UTF-8 encoded. (Erik Hatcher)

* SOLR-7929: SimplePostTool (also bin/post) -filetypes "*" now works properly in

'web' mode (Erik Hatcher)
* SOLR-7978: Fixed example/files update-script.js to be Java 7 and 8 compatible.
(Erik Hatcher)

* SOLR-7988: SolrJ could not make requests to handlers with '/admin/' prefix (noble
, ludovic Boutros)

* SOLR-7990: Use of timeAllowed can cause incomplete filters to be cached and

incorrect
results to be returned on subsequent requests. (Erick Erickson, yonik)

* SOLR-8041: Fix VelocityResponseWriter's $resource.get(key,baseName,locale) to use

specified locale.
(Erik Hatcher)

================== 5.3.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.2.11.v20150529

Upgrading from Solr 5.2

-----------------------

* SolrJ's CollectionAdminRequest class is now marked as abstract. Use one of its

concrete
sub-classes instead.

* Solr no longer supports forcefully unlocking an index.

This is no longer supported by the underlying Lucene locking
framework. The setting in solrconfig.xml has no effect anymore.
Background: If you use native lock factory, unlocking should
not be needed, because the locks are cleared after process
shutdown automatically by the operating system. If you are
using simple lock factory (not recommended) or hdfs lock
factory, you may need to manually unlock by deleting the lock
file from filesystem / HDFS.

* The zkCredientialsProvider element in solrcloud section of solr.xml is now

deprecated.
Use the correct spelling (zkCredentialsProvider) instead.

* class TransformerWithContext is deprecated . Use DocTransformer directly

* The "name" parameter in ADDREPLICA Collections API call has be deprecated. One
cannot specify
the core name for a replica. See SOLR-7499 for more info.

* The ShardHandler interface has changed. The interface used to provide a

`checkDistributed` function which doubled up in purpose to determine if the
request is distributed, and to prepare for distributed requests. This
unfortunately
 meant that the object had to be instantiated even when the request is not
distributed. The task of initially determining if the request is distributed
is now done by SearchHandler using the distrib/shards parameters, and a
ShardHandler object is created only if the request is distributed. The interface
now has a `prepDistributed` function instead of the `checkDistributed` function,
which can then be used to prepare for the distributed request. Users with custom
ShardHandler implementations would need to modify their code to this effect.

* The system property "solr.solrxml.location" is not supported any more. Now,

solr.xml is first
looked up in zookeeper, and if not found, fallback to SOLR_HOME. See SOLR-7735
for more info.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-7724: SolrJ now supports parsing the output of the clustering component.
(Alessandro Benedetti via Dawid Weiss)

* SOLR-7389: Expose znodeVersion property for each of the collections returned for
the clusterstatus
operation in the collections API (Marius Grama via shalin)

* SOLR-7622: A DocTransformer can now request fields from the SolrIndexSearcher

that are not
necessarily returned in the file SolrDocument by returning a list of fields from
DocTransformer#getExtraRequestFields (ryan)

* SOLR-7458: Expose HDFS Block Locality Metrics via JMX (Mike Drob via Mark Miller)

* SOLR-7676: Faceting on nested objects / Block-join faceting with the new JSON
Facet API.
Example: Assuming books with nested pages and an input domain of pages, the
following
will switch the domain to books before faceting on the author field:
authors:{ type:terms, field:author, domain:{toParent:"type:book"} }
(yonik)

* SOLR-7668: Add 'port' tag support in replica placement rules (Adam McElwee, Noble
Paul)

* SOLR-5886: Response for an async call is now stored in zk so that it can be

returned by the REQUESTSTATUS API.
Also, the number of stored (failed and successful) responses are now restricted
to 10,000 each as a safety net.
(Anshum Gupta)

* SOLR-7639: MoreLikeThis QParser now supports all options provided by the MLT
Handler i.e. mintf, mindf,
minwl, maxwl, maxqt, and maxntp.

* SOLR-7182: Make the Schema-API a first class citizen of SolrJ. The new
SchemaRequest and its inner
classes can be used to make requests to the Schema API. (Sven Windisch, Marius
Grama via shalin)
* SOLR-7651: New response format added wt=smile (noble)

* SOLR-4212: SOLR-6353: Let facet queries and facet ranges hang off of pivots.
Example:
facet.range={!tag=r1}price&facet.query={!tag=q1}somequery&facet.pivot={!range=r1
query=q1}category,manufacturer
(Steve Molloy, hossman, shalin)

* SOLR-7742: Support for Immutable ConfigSets (Gregory Chanan)

* SOLR-2522: new two argument option for the existing field() function; picks the
min/max value of a
docValues field to use as a ValueSource: "field(field_name,min)" and
"field(field_name,max)" (hossman)

* SOLR-6234: Scoring for query time join (Mikhail Khludnev)

* SOLR-5882: score local parameter for block join query parser {!parent} (Andrey
Kudryavtsev, Mikhail Khludnev)

* SOLR-7799: Added includeIndexFieldFlags (backwards compatible default is true) to

/admin/luke.
When there are many fields in the index, setting this flag to false can
dramatically speed up requests. (ehatcher)

* SOLR-7769: Add bin/post -p alias for -port parameter. (ehatcher)

* SOLR-7766: support creation of a coreless collection via createNodeSet=EMPTY

(Christine Poerschke)

* SOLR-7849: Solr-managed inter-node authentication when authentication enabled

(Noble Paul)

* SOLR-7220: Nested C-style comments in queries. (yonik)

* SOLR-7757: Improved security framework where security components can be

edited/reloaded, Solr
now watches /security.json. Components can choose to make their config editable
(Noble Paul, Anshum Gupta, Ishan Chattopadhyaya)

* SOLR-7838: An authorizationPlugin interface where the access control rules are

stored/managed in
ZooKeeper (Noble Paul, Anshum Gupta, Ishan Chattopadhyaya)

* SOLR-7837: An AuthenticationPlugin which implements the HTTP BasicAuth protocol

and stores credentials
securely in ZooKeeper (Noble Paul, Anshum Gupta,Ishan Chattopadhyaya)

Bug Fixes
----------------------

* SOLR-7361: Slow loading SolrCores should not hold up all other SolrCores that
have finished loading from serving
requests. (Mark Miller, Timothy Potter, Ramkumar Aiyengar)

* SOLR-4506: Clean-up old (unused) index directories in the background after

initializing a new index;
previously, Solr would leave old index.yyyyMMddHHmmssSSS directories left behind
after failed recoveries
in the data directory, which unnecessarily consumes disk space. (Mark Miller,
Timothy Potter)

* SOLR-7108: Change default query used by /admin/ping to not rely on other

parameters such as query parser or
default field. (ehatcher)

* SOLR-6835: ReRankQueryParserPlugin checks now whether the reRankQuery parameter

is present and not empty.
(帅广应, Marius Grama via shalin)

* SOLR-7566: Search requests should return the shard name that is down. (Marius
Grama, shalin)

* SOLR-7675: Add missing _root_ field to managed-schema template so that the

default data driven
config set can index nested documents by default. (yonik)

* SOLR-7635: Limit lsof port check in bin/solr to just listening ports

(Upayavira, Ramkumar Aiyengar)

* SOLR-7091: Nested documents with unknown fields don't work in schemaless mode.
(Steve Rowe)

* SOLR-7682: Schema API: add-copy-field should accept the maxChars parameter.

(Steve Rowe)

* SOLR-7693: Fix the bin/solr -e cloud example to work if lsof is not installed
on the local machine by waiting for 10 seconds before starting the second node.
(hossman, Timothy Potter)

* SOLR-7689: ReRankQuery rewrite method can change the QueryResultKey causing cache
misses.
(Emad Nashed, Yonik Seeley, Joel Bernstein)

* SOLR-7697: Schema API doesn't take class or luceneMatchVersion attributes into

account for the analyzer when adding a new field type. (Marius Grama, Steve Rowe)

* SOLR-7679: Schema API doesn't take similarity attribute into account when adding
field types. (Marius Grama, Steve Rowe)

* SOLR-7664: Throw correct exception (RemoteSolrException) on receiving a HTTP 413.

(Ramkumar Aiyengar, Eirik Lygre)

* SOLR-6686: facet.threads can return wrong results when using facet.prefix

multiple
times on same field. (Michael Ryan, Tim Underwood via shalin)

* SOLR-7673: Race condition in shard splitting can cause operation to hang

indefinitely
or sub-shards to never become active. (shalin)

* SOLR-7741: Add missing fields to SolrIndexerConfig.toMap

(Mike Drob, Christine Poerschke via Ramkumar Aiyengar)

* SOLR-7748: Fix bin/solr to start on IBM J9. (Shai Erera)

* SOLR-7143: MoreLikeThis Query parser should handle multiple field names

 (Jens Wille, Anshum Gupta)

* SOLR-7132: The Collections API ADDREPLICA command property.name is not reflected

in the clusterstate until after Solr restarts (Erick Erickson)

* SOLR-7172: addreplica API fails with incorrect error msg "cannot create
collection"
(Erick Erickson)

* SOLR-7705: CoreAdminHandler Unload no longer handles null core name and throws
NPE
instead of a bad request error. (John Call, Edward Ribeiro via shalin)

* SOLR-7529: CoreAdminHandler Reload throws NPE on null core name instead of a bad
request error. (Jellyfrog, Edward Ribeiro via shalin)

* SOLR-7781: JSON Facet API: Terms facet on string/text fields with sub-facets
caused
a bug that resulted in filter cache lookup misses as well as the filter cache
exceeding it's configured size. (yonik)

* SOLR-7810: map-reduce contrib script to set classpath for convenience refers to

example
rather than server. (Mark Miller)

* SOLR-7765: Hardened the behavior of TokenizerChain when null arguments are used
in constructor.
This prevents NPEs in some code paths. (Konstantin Gribov, hossman)

* SOLR-7829: Fixed a bug in distributed pivot faceting that could result in a

facet.missing=true count
which was lower then the correct count if facet.sort=index and
facet.pivot.mincount > 1 (hossman)

* SOLR-7842: ZK connection loss or session expiry events should not fire config
directory listeners.
(noble, shalin)

* SOLR-6357: Allow delete documents by doing a score join query. (Mikhail Khludnev,
Timothy Potter)

* SOLR-7756: Fixed ExactStatsCache and LRUStatsCache to not throw an NPE when a

term is not present on a shard.
(Varun Thacker, Anshum Gupta)

* SOLR-7818: Fixed distributed stats to be calculated for all the query terms.
Earlier the stats were calculated with
the terms that are present in the last shard of a distributed request. (Varun
Thacker, Anshum Gupta)

* SOLR-7866: VersionInfo caused an unhandled NPE when trying to determine the max
value for the
version field. (Timothy Potter)

* SOLR-7666 (and linked tickets): Many fixes to AngularJS Admin UI bringing it

close to feature
parity with existing UI. (Upayavira)

* SOLR-7908: SegmentsInfoRequestHandler gets a ref counted IndexWriter and does not

properly release it.
(Mark Miller, shalin)

* SOLR-7921: The techproducts example fails when running in a directory that

contains spaces.
(Ishan Chattopadhyaya via Timothy Potter)

* SOLR-7934: SolrCLI masks underlying cause of create collection failure. (Timothy

Potter)

Optimizations
----------------------

* SOLR-7660: Avoid redundant 'exists' calls made to ZK while fetching cluster state
updates. (shalin)

* SOLR-7714: Reduce SearchHandler's use of ShardHandler objects across shards in a

search,
from one for each shard and the federator, to just one for the federator.
(Christine Poerschke via Ramkumar Aiyengar)

* SOLR-7751: Minor optimizations to QueryComponent.process (reduce eager

instantiations,
cache method calls) (Christine Poerschke via Ramkumar Aiyengar)

* SOLR-7455: Terms facets with the JSON Facet API now defer calculating non-sorting
stats
until a second phase, after the top N facets are found. This improves performance
proportional to the number of non-sorting statistics being calculated in addition
to
the number of buckets and domain documents.
For Example: The facet request {type:terms, field:field1, facet:
{x:"unique(field2)"}}
saw a 7x improvement when field1 and 1M unique terms and field2 had 1000 unique
terms.
(yonik)

* SOLR-7840: ZkStateReader.updateClusterState fetches watched collections twice

from ZK. (shalin)

* SOLR-7875: Speedup SolrQueryTimeoutImpl. Avoid setting a timeout time when

timeAllowed
parameter is not set. (Tomás Fernández Löbbe)

Other Changes
----------------------

* SOLR-7787: Removed fastutil and java-hll dependency, integrated HyperLogLog from

java-hll
into Solr core. (Dawid Weiss)

* SOLR-7595: Allow method chaining for all CollectionAdminRequests in Solrj.

(shalin)

* SOLR-7146: MiniSolrCloudCluster based tests can fail with ZooKeeperException

NoNode for /live_nodes.
(Vamsee Yarlagadda via shalin)
* SOLR-7590: Finish and improve MDC context logging support. (Mark Miller)

* SOLR-7599: Remove cruft from SolrCloud tests. (shalin)

* SOLR-7636: CLUSTERSTATUS API is executed at CollectionsHandler (noble)

* LUCENE-6508: Remove ability to forcefully unlock an index.

This is no longer supported by the underlying Lucene locking
framework. (Uwe Schindler, Mike McCandless, Robert Muir)

* SOLR-3719: Add as-you-type "instant search" to example/files /browse.

(Esther Quansah, ehatcher)

* SOLR-7645: Remove explicitly defined request handlers from example and test
solrconfig's that are
already defined implicitly, such as /admin/ping, /admin/system, and several
others. (ehatcher)

* SOLR-7603: Fix test only bug in UpdateRequestProcessorFactoryTest (hossman)

* SOLR-7634: Upgrade Jetty to 9.2.11.v20150529 (Bill Bell, shalin)

* SOLR-7659: Rename releaseCommitPointAndExtendReserve in DirectoryFileStream

to extendReserveAndReleaseCommitPoint, and reverse the code to match.
(shalin, Shawn Heisey)

* SOLR-7624: Add correct spelling (zkCredentialsProvider) as an alternative to

zkCredientialsProvider element in solrcloud section of solr.xml.
(Xu Zhang, Per Steffensen, Ramkumar Aiyengar, Mark Miller)

* SOLR-7619: Fix SegmentsInfoRequestHandlerTest when more than one segment is

created.
(Ramkumar Aiyengar, Steve Rowe)

* SOLR-7678: Switch RTimer to use nanoTime (improves accuracy of QTime, and other
times
returned by Solr handlers) (Ramkumar Aiyengar)

* SOLR-7680: Use POST instead of GET when finding versions for mismatches with
CloudInspectUtil for tests (Ramkumar Aiyengar)

* SOLR-7665: deprecate the class TransformerWithContext (noble)

* SOLR-7629: Have RulesTest consider disk space limitations of where the test is
being run (Christine Poerschke via Ramkumar Aiyengar)

* SOLR-7499: The "name" parameter in ADDREPLICA Collections API call has be

deprecated. One cannot specify
the core name for a replica (Varun Thacker, noble, Erick Erickson)

* SOLR-7711: Correct initial capacity for the list that holds the default
components for the SearchHandler
(Christine Poerschke via Varun Thacker)

* SOLR-7485: Replace shards.info occurrences with ShardParams.SHARDS_INFO

(Christine Poerschke via Ramkumar Aiyengar)

* SOLR-7710: Replace async occurrences with CommonAdminParams.ASYNC

(Christine Poerschke, Ramkumar Aiyengar)
* SOLR-7712: fixed test to account for aggregate floating point precision loss
(hossman)

* SOLR-7740: Fix typo bug with TestConfigOverlay (Christine Poerschke via Ramkumar
Aiyengar)

* SOLR-7750: Change TestConfig.testDefaults to cover all SolrIndexConfig fields

(Christine Poerschke via Ramkumar Aiyengar)

* SOLR-7703: Authentication plugin is now loaded using the ResourceLoader.

(Avi Digmi via Anshum Gupta)

* SOLR-7800: JSON Facet API: the avg() facet function now skips missing values
rather than treating them as a 0 value. The def() function can be used to
treat missing values as 0 if that is desired.
Example: facet:{ mean:"avg(def(myfield,0))" }

* SOLR-7805: Update Kite Morphlines to 1.1.0 (Mark Miller)

* SOLR-7803: Prevent class loading deadlock in TrieDateField; refactor date

formatting and parsing out of TrieDateField and move to static utility class
DateFormatUtil. (Markus Heiden, Uwe Schindler)

* SOLR-7825: Forbid all usages of log4j and java.util.logging classes in Solr

except
classes which are specific to logging implementations. Remove accidental usage of
log4j
logger from a few places. The default log level for org.apache.zookeeper is
changed from
ERROR to WARN for zkcli.{sh,cmd} only.
(Oliver Schrenk, Tim Potter, Uwe Schindler, shalin)

* SOLR-7735: Look for solr.xml in Zookeeper by default in SolrCloud mode. If not

found, it will be loaded
from $SOLR_HOME/solr.xml as before. Sysprop solr.solrxml.location is now gone.
(janhoy)

* SOLR-7227: Ship Solr with the Web application directory exploded into
server/solr-webapp, solr.war is no longer included in the distribution
bundles. (Timothy Potter, Uwe Schindler)

* SOLR-6625: Enable registering interceptors for the calls made using HttpClient
and make the
request object available at the interceptor context ( Ishan Chattopadhyay,
Gregory Chanan, noble, Anshum Gupta)

* SOLR-5022: On Java 7 raise permgen for running tests. (Uwe Schindler)

* SOLR-7823: TestMiniSolrCloudCluster.testCollectionCreateSearchDelete async

collection-creation (sometimes)
(Christine Poerschke)

* SOLR-7854: Remove unused ZkStateReader.updateClusterState(false) method. (Scott

Blum via shalin)

* SOLR-7863: Lowercase the CLUSTERPROP command in ZkCLI for consistency, print

error for unknown cmd (janhoy)
* SOLR-7832: bin/post now allows either -url or -c, rather than requiring both.
(ehatcher)

* SOLR-7847: Implement run example logic in Java instead of OS-specific scripts in

bin/solr and bin\solr.cmd (Timothy Potter)

* SOLR-7877: TestAuthenticationFramework.testBasics to preserve/restore the

original request(Username|Password)
(Christine Poerschke)

* SOLR-7900: example/files improvements - added language detection and faceting,

added title field, relocated .js files.
(Esther Quansah and Erik Hatcher)

================== 5.2.1 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.2.10.v20150310

Detailed Change List

----------------------

Bug Fixes
----------------------

* SOLR-7588: Fix javascript bug introduced by SOLR-7409 that breaks the

dataimport screen in the admin UI. (Bill Bell via Shawn Heisey)

* SOLR-7616: Faceting on a numeric field with a unique() subfacet function on

another numeric field
can result in incorrect results or an exception. (yonik)

* SOLR-7518: New Facet Module should respect shards.tolerant and process all non-
failing shards
instead of throwing an exception. (yonik)

* SOLR-7574: A request with a json content type but no body caused a null pointer
exception (yonik)

* SOLR-7512: SolrOutputFormat creates an invalid solr.xml in the solr home zip for
MapReduceIndexerTool.
(Mark Miller, Adam McElwee)

* SOLR-7652: Fix example/files update-script.js to work with Java 7 (ehatcher)

* SOLR-7638: Fix new (Angular-based) admin UI Cloud pane (Upayavira via ehatcher)

* SOLR-7655: The DefaultSolrHighlighter since 5.0 was determining if payloads were

present in a way
that was slow, especially when lots of fields were highlighted. It's now fast.
(David Smiley)

* SOLR-7493: Requests aren't distributed evenly if the collection isn't present

locally.
(Jeff Wartes, shalin)

Other Changes
----------------------

* SOLR-7623: Fix regression from SOLR-7484 that made it impossible to override

SolrDispatchFilter#execute() and SolrDispatchFilter#sendError(). You can now
override these
functions in HttpSolrCall. (ryan)

* SOLR-7648: Expose remote IP and Host via the AuthorizationContext to be used by

the authorization plugin.
(Ishan Chattopadhyaya via Anshum Gupta)

================== 5.2.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 9.2.10.v20150310

Upgrading from Solr 5.1

-----------------------

* A bug was introduced in Solr 4.10 that caused index time document boosts to
trigger excessive field
boosts in multivalued fields -- the result being that some field norms might be
excessively large.
This bug has now been fixed, but users of document boosts are strongly encouraged
to re-index.
See SOLR-7335 for more details.

* The Slice and Replica classes have been changed to use State enums instead of
string constants
to track the respective stats. Advanced users with client code manipulating
these objects will
need to update their code accordingly. See SOLR-7325 and SOLR-7336 for more
info.

* Solr has internally been upgraded to use Jetty 9. See SOLR-4839 for full details,
but there are
a few key details all Solr users should know when upgrading:

- It is no longer possible to run "java -jar start.jar" from inside the server
directory.
The bin/solr script is the only supported way to run Solr. This is necessary to
support
HTTP and HTTPS modules in Jetty which can be selectively enabled by the
bin/solr scripts.
In case you have a pressing need to run solr the old way, you can run
"java -jar start.jar --module=http" to get the same behavior as before.

- The way SSL support is configured has been changed. Before this release,
the SOLR_SSL_OPTS property configured in solr.in.sh (linux/mac) or solr.in.cmd
(windows)
was used to enable/disable SSL but starting in 5.2.0, new properties named as
SOLR_SSL_KEY_STORE, SOLR_SSL_KEY_STORE_PASSWORD, SOLR_SSL_TRUST_STORE,
SOLR_SSL_TRUST_STORE_PASSWORD, SOLR_SSL_NEED_CLIENT_AUTH and
SOLR_SSL_WANT_CLIENT_AUTH
have been introduced. The bin/solr scripts configure the SOLR_SSL_OPTS property
automatically based on the above new properties.
You should *not* configure the SOLR_SSL_OPTS property directly inside solr.in.
{sh,cmd}.

- Support for SOLR_SSL_PORT property has been removed. Instead use the regular
SOLR_PORT
property or specify the port while invoking the bin/solr script using the "-p"
switch.

- Furthermore, it is now possible to configure the HTTP client with

different SSL properties than the ones used for Jetty using the same files.

- Please refer to the "Enabling SSL" section in the Solr Reference Guide for
complete details.

* Support for pathPrefix has been completely removed from Solr. Since 5.0, Solr no
longer officially
supports being run as a webapp but allowed users to play around with the web.xml
to have a path prefix.
That would no longer be true. See SOLR-7500 for more info.

* The package structure under org.apache.solr.client.solrj.io has been changed to

support
the Streaming Expression Language (SOLR-7377). Any code written with the 5.1
Streaming API will have to
be updated to reflect these changes.

* Merge Policy's "noCFSRatio" is no longer set based on <useCompoundFile> element

in the indexConfig section
of solrconfig.xml. This means that Solr will start using Lucene's default for MP
"noCFSRatio", with this
new default Solr will decide if a segment should use cfs or not based on the size
of the segment in relation
the size of the complete index. For TieredMergePolicy for example (current
default), segments will use cfs
if they are less than 10% of the index, otherwise cfs is disabled. Old values for
this setting
(1.0 for useCompoundFile=true and 0.0 for useCompoundFile=false) as well as any
other value can be set
inside the <mergePolicy> element in solrconfig.xml. <useCompoundFile> will only
apply to newly created
segments. See SOLR-7463.

Detailed Change List

----------------------
New Features
----------------------

* SOLR-6637: Restore a Solr core from a backed up index.

Restore API Example -
http://localhost:8983/solr/techproducts/replication?
command=restore&name=backup_name
Restore Status API Example -
http://localhost:8983/solr/techproducts/replication?command=restorestatus
(Varun Thacker, noble, shalin)

* SOLR-7241, SOLR-7263, SOLR-7279, SOLR-7300, SOLR-7396, SOLR-7397, SOLR-7492:

Admin UI - Refactoring using AngularJS. More functionality moving the Admin
UI to Angular JS (Upayavira via Erick)

* SOLR-7372: Limit memory consumed by LRUCache with a new 'maxRamMB' config

parameter.
(yonik, shalin)

* SOLR-7376: Return raw XML or JSON (in the appropriate writer) using
DocumentTransformers.
?fl=id,name,json_s:[json],xml_s:[xml] (ryan)

* SOLR-7422: Optional flatter form for the JSON Facet API via a "type" parameter:
top_authors : { type:terms, field:author, limit:5 } is equivalent to
top_authors : { terms : { field:author, limit:5 } }
(yonik)

* SOLR-7176: zkcli script can perfrom the CLUSTERPROP command without a running
Solr cluster
(Hrishikesh Gadre, Per Steffensen, Noble Paul)

* SOLR-7417: JSON Facet API - unique() is now implemented for numeric and date
fields.
(yonik)

* SOLR-7406: Add a new "facet.range.method" parameter to let users choose how to do

range
faceting between an implementation based on filters (previous algorithm, using
"facet.range.method=filter") or DocValues ("facet.range.method=dv").
Input parameters and output of both methods are the same. (Tomás Fernández Löbbe)

* SOLR-7473: Facet Module (Json Facet API) range faceting now supports the
"mincount"
parameter in range facets to supress buckets less than that count. The default
for "mincount" remains 0 for range faceting.
Example: prices:{ type:range, field:price, mincount:1, start:0, end:100, gap:10 }
(yonik)

* SOLR-7437: Make HDFS transaction log replication factor configurable. (Mark

Miller)

* SOLR-7477: Multi-select faceting support for the Facet Module via the
"excludeTags"
parameter which disregards any matching tagged filters for that facet. Example:
& q=shoes
& fq={!tag=COLOR}color:blue
& json.facet={ colors:{type:terms, field:color, excludeTags=COLOR} }
(yonik)
* SOLR-7231: DIH-TikaEntityprocessor, create lat-lon field from Metadata
(Tim Allison via Noble Paul)

* SOLR-6220: Rule Based Replica Assignment during collection, shard creation

and replica creation (Noble Paul)

* SOLR-6968: New 'cardinality' option for stats.field, uses HyperLogLog to

efficiently
estimate the cardinality of a field w/bounded RAM. (hossman)

* SOLR-4392: Make it possible to specify AES encrypted password in dataconfig.xml

(Noble Paul)

* SOLR-7461: stats.field now supports individual local params for 'countDistinct'

and 'distinctValues'.
'calcdistinct' is still supported as an alias for both options (hossman)

* SOLR-7522: Facet Module - Implement field/terms faceting over single-valued

numeric fields. (yonik)

* SOLR-7275: Authorization framework for Solr. It defines an interface and a

mechanism to create,
load, and use an Authorization plugin. (Noble Paul, Ishan Chattopadhyaya, Anshum
Gupta)

* SOLR-7377: Solr Streaming Expressions (Dennis Gove, Joel Bernstein, Steven Bower)

* SOLR-7553: Facet Analytics Module: new "hll" function that uses HyperLogLog to
calculate
distributed cardinality. The original "unique" function is still available.
Example: json.facet={ numProducts : "hll(product_id)" }
(yonik)

* SOLR-7546: bin/post (and SimplePostTool in -Dauto=yes mode) now sends rather than
skips files
without a known content type, as "application/octet-stream", provided it still is
in the
allowed filetypes setting. (ehatcher)

* SOLR-7274: Pluggable authentication module in Solr. This defines an interface and

a mechanism to create,
load, and use an Authentication plugin. (Noble Paul, Ishan Chattopadhyaya,
Gregory Chanan, Anshum Gupta)

* SOLR-7379: (experimental) New spatial RptWithGeometrySpatialField, based on

CompositeSpatialStrategy,
which blends RPT indexes for speed with serialized geometry for accuracy.
Includes a Lucene segment based
in-memory shape cache. (David Smiley)

* SOLR-7465, SOLR-7610: New file indexing example, under example/files. (Esther

Quansah, Erik Hatcher)

* SOLR-7468: Kerberos authenticaion plugin for Solr. This would allow running a
Kerberized Solr.
(Noble Paul, Ishan Chattopadhyaya, Gregory Chanan, Anshum Gupta)

Bug Fixes
----------------------

* SOLR-6709: Fix QueryResponse to deal with the "expanded" section when using the
XMLResponseParser
(Varun Thacker, Joel Bernstein)

* SOLR-7066: autoAddReplicas feature has bug when selecting replacement nodes.

(Mark Miller)

* SOLR-7370: FSHDFSUtils#recoverFileLease tries to recover the lease every one

second after
the first four second wait. (Mark Miller)

* SOLR-7369: AngularJS UI insufficient URLDecoding in cloud/tree view (janhoy)

* SOLR-7380: SearchHandler should not try to load runtime components in inform()

(Noble Paul)

* SOLR-7385: The clusterstatus API now returns the config set used to create a
collection
inside a 'configName' key. (Shai Erera, shalin)

* SOLR-7401: Fixed a NullPointerException when concurrently creating and deleting

collections,
while accessing other collections. (Shai Erera)

* SOLR-7412: Fixed range.facet.other parameter for distributed requests.

(Will Miller, Tomás Fernández Löbbe)

* SOLR-6087: SolrIndexSearcher makes no DelegatingCollector.finish() call when

IndexSearcher
throws an expected exception. (Christine Poerschke via shalin)

* SOLR-7420: Overseer stats are not reset on loss of ZK connection. (Jessica Cheng,
shalin)

* SOLR-7392: Fix SOLR_JAVA_MEM and SOLR_OPTS customizations in solr.in.sh being

ignored
(Ramkumar Aiyengar, Ere Maijala)

* SOLR-7426: SolrConfig#getConfigOverlay does not clean up it's resources. (Mark

Miller)

* SOLR-6665: ZkController.publishAndWaitForDownStates can return before all local

cores are
marked as 'down' if multiple replicas with the same core name exist in the
cluster.
(shalin)

* SOLR-7418: Check and raise a SolrException instead of an NPE when an invalid doc
id is sent
to the MLTQParser. (Anshum Gupta)

* SOLR-7443: Implemented range faceting over date fields in the new facet module
(JSON Facet API). (yonik)

* SOLR-7440: DebugComponent does not return the right requestPurpose for pivot
facet refinements.
(shalin)
* SOLR-7408: Listeners set by SolrCores on config directories in ZK could be
removed if collections
are created/deleted in paralle against the same config set. (Shai Erera, Anshum
Gupta)

* SOLR-7450: Fix edge case which could cause `bin/solr stop` to hang forever
(Ramkumar Aiyengar)

* SOLR-7157: initParams must support tags other than appends, defaults and,
invariants (Noble Paul)

* SOLR-7387: Facet Module - distributed search didn't work when sorting terms
facet by min, max, avg, or unique functions. (yonik)

* SOLR-7469: Fix check-licenses to correctly detect if start.jar.sha1 is incorrect

(hossman)

* SOLR-7449: solr/server/etc/jetty-https-ssl.xml hard codes the key store file and

password rather
than pulling them from the sysprops defined in solr/bin/solr.in.{sh,cmd}

* SOLR-7470: Fix sample data to eliminate file order dependency for successful
indexing, also
fixed SolrCloudExampleTest to help catch this in the future. (hossman)

* SOLR-7478: UpdateLog#close shuts down it's executor with interrupts before

running it's close logic,
possibly preventing a clean close. (Mark Miller)

* SOLR-7494: Facet Module - unique() facet function was wildly inaccurate for high
cardinality
fields. (Andy Crossen, yonik)

* SOLR-7502: start script should not try to create configset for .system collection
(Noble Paul)

* SOLR-7514: SolrClient.getByIds fails with ClassCastException (Tom Farnworth,

Ramkumar Aiyengar)

* SOLR-7531: config API shows a few keys merged together (Noble Paul)

* SOLR-7542: Schema API: Can't remove single dynamic copy field directive
(Steve Rowe)

* SOLR-7472: SortingResponseWriter does not log fl parameters that don't exist.

(Joel Bernstein)

* SOLR-7545: Honour SOLR_HOST parameter with bin/solr{,.cmd}

(Ishan Chattopadhyaya via Ramkumar Aiyengar)

* SOLR-7503: Recovery after ZK session expiration should happen in parallel for all
cores
using the thread-pool managed by ZkContainer instead of a single thread.
(Jessica Cheng Mallet, Timothy Potter, shalin, Mark Miller)

* SOLR-7335: Fix doc boosts to no longer be multiplied in each field value in

multivalued fields that
are not used in copyFields (Shingo Sasaki via hossman)
* SOLR-7585: Fix NoSuchElementException in LFUCache resulting from heavy writes
making concurrent put() calls. (Maciej Zasada via Shawn Heisey)

* SOLR-7587: Seeding bucket versions from index when the firstSearcher event fires
has a race condition
that leads to an infinite wait on VersionInfo's ReentrantReadWriteLock because
the read-lock acquired
during a commit cannot be upgraded to a write-lock needed to block updates;
solution is to move the
call out of the firstSearcher event path and into the SolrCore constructor.
(Timothy Potter)

* SOLR-7625: Ensure that the max value for seeding version buckets is updated after
recovery even if
the UpdateLog is not replayed. (Timothy Potter)

* SOLR-7610: Fix VelocityResponseWriter's $resource.locale to accurately report

locale in use.
(ehatcher)

* SOLR-7614: Distributed pivot facet refinement was broken due to a single

correlation counter
used across multiple requests as if it was private to each request. (yonik)

Optimizations
----------------------

* SOLR-7324: IndexFetcher does not need to call isIndexStale if full copy is

already needed
(Stephan Lagraulet via Varun Thacker)

* SOLR-7547: Short circuit SolrDisptachFilter for static content request. Right now
it creates
a new HttpSolrCall object and tries to process it. (Anshum Gupta)

* SOLR-7333: Make the poll queue time a leader uses when distributing updates to
replicas
configurable and use knowledge that a batch is being processed to poll
efficiently.
(Timothy Potter)

* SOLR-7332: Initialize the highest value for all version buckets with the max
value from
the index or recent updates to avoid unnecessary lookups to the index to check
for reordered
updates when processing new documents. (Timothy Potter, yonik)

* SOLR-5855: DefaultSolrHighlighter now re-uses the document's term vectors

instance when highlighting
more than one field. Applies to the standard and FVH highlighters. (David Smiley,
Daniel Debray)

Other Changes
----------------------

* SOLR-6865: Upgrade HttpClient to 4.4.1 (Shawn Heisey)

* SOLR-7358: TestRestoreCore fails in Windows (Ishan Chattopadhyaya via Varun
Thacker)

* SOLR-7371: Make DocSet implement Accountable to estimate memory usage. (yonik,

shalin)

* SOLR-7381: Improve logging by adding node name in MDC in SolrCloud mode and
adding MDC to
all thread pools. A new MDCAwareThreadPoolExecutor is introduced and usages of
Executors#newFixedThreadPool, #newSingleThreadExecutor, #newCachedThreadPool as
well as
ThreadPoolExecutor directly is now forbidden in Solr. MDC keys are now exposed in
thread
names automatically so that a thread dump can give hints on what the thread was
doing.
Uncaught exceptions thrown by tasks in the pool are logged along with submitter's
stack trace.
(shalin)

* SOLR-7384: Fix spurious failures in FullSolrCloudDistribCmdsTest. (shalin)

* SOLR-6692: Default highlighter changes:

- hl.maxAnalyzedChars now applies cumulatively on a multi-valied field.
- fragment ranking on a multi-valued field should be more relevant.
- hl.usePhraseHighlighter is now toggleable on a per-field basis.
- Much more extensible (get values from another source; return snippet scores and
offsets).
- When using hl.maxMultiValuedToMatch with hl.preserveMulti, only count matched
snippets.
(David Smiley)

* SOLR-6886: Removed redundant size check and added missing calls to

DelegatingCollection.finish inside Grouping code. (Christine Poerschke via
shalin)

* SOLR-7421: RecoveryAfterSoftCommitTest fails frequently on Jenkins due to full

index
replication taking longer than 30 seconds. (Timothy Potter, shalin)

* SOLR-7081: Add new test case to test if create/delete/re-create collections work.

(Christine Poerschke via Ramkumar Aiyengar)

* SOLR-7467: Upgrade t-digest to 3.1 (hossman)

* SOLR-7471: Stop requiring docValues for interval faceting (Tomás Fernández Löbbe)

* SOLR-7391: Use a time based expiration cache for one off HDFS FileSystem
instances.
(Mark Miller)

* SOLR-5213: Log when shard splitting unexpectedly leads to documents going to

no or multiple shards (Christine Poerschke, Ramkumar Aiyengar)

* SOLR-7425: Improve MDC based logging format. (Mark Miller)

* SOLR-4839: Upgrade Jetty to 9.2.10.v20150310 and restlet-jee to 2.3.0

(Bill Bell, Timothy Potter, Uwe Schindler, Mark Miller, Steve Rowe, Steve Davids,
shalin)
* SOLR-7457: Make DirectoryFactory publishing MBeanInfo extensible.
(Mike Drob via Mark Miller)

* SOLR-7325: Slice.getState() now returns a State enum instead of a String. This

helps
clarify the states a Slice can be in, as well comparing the state of a Slice.
(Shai Erera)

* SOLR-7336: Added Replica.getState() and removed ZkStateReader state-related

constants.
You should use Replica.State to compare a replica's state. (Shai Erera)

* SOLR-7487: Fix check-example-lucene-match-version Ant task to check

luceneMatchVersion
in solr/server/solr/configsets instead of example and harden error checking /
validation
logic. (hossman, Timothy Potter)

* SOLR-7409: When there are multiple dataimport handlers defined, the admin UI
was listing them in a random order. Now they are sorted in a natural order
that handles numbers properly. (Jellyfrog via Shawn Heisey)

* SOLR-7484: Refactor SolrDispatchFilter to extract all Solr specific

implementation detail
to HttpSolrCall and also extract methods from within the current SDF.doFilter(..)
logic
making things easier to manage. HttpSolrCall converts the processing to a 3-step
process
i.e. Construct, Init, and Call so the context of the request would be available
after Init
and before the actual call operation. (Anshum Gupta, Noble Paul)

* SOLR-6878: Allow symmetric lists of synonyms to be added using the managed

synonym REST
API to support legacy expand=true type mappings; previously the API only allowed
adding
explicit mappings, with this feature you can now add a list and have the mappings
expanded when the update is applied (Timothy Potter, Vitaliy Zhovtyuk, hossman)

* SOLR-7102: bin/solr should activate cloud mode if ZK_HOST is set (Timothy Potter)

* SOLR-7500: Remove pathPrefix from SolrDispatchFilter as Solr no longer runs as a

part
of a bigger webapp. (Anshum Gupta)

* SOLR-7243: CloudSolrClient was always returning SERVER_ERROR for exceptions,

even when a more relevant ErrorCode was available, via SolrException. Now
the actual ErrorCode is used when available.
(Hrishikesh Gadre via Shawn Heisey)

* SOLR-7544: CollectionsHandler refactored to be more modular (Noble Paul)

* SOLR-7532: Removed occurrences of the unused 'commitIntervalLowerBound' property

for
updateHandler elements from Solr configuration. (Marius Grama via shalin)

* SOLR-7541: Removed CollectionsHandler#createNodeIfNotExists. All calls made to

this method now call
ZkCmdExecutor#ensureExists as they were doing the same thing. Also
ZkCmdExecutor#ensureExists now respects the
CreateMode passed to it. (Varun Thacker)

* SOLR-6820: Make the number of version buckets used by the UpdateLog configurable
as
increasing beyond the default 256 has been shown to help with high volume
indexing
performance in SolrCloud; helps overcome a limitation where Lucene uses the
request
thread to perform expensive index housekeeping work. (Mark Miller, yonik, Timothy
Potter)

* SOLR-7463: Stop forcing MergePolicy's "NoCFSRatio" based on the IWC

"useCompoundFile" configuration
(Tomás Fernández Löbbe)

* SOLR-7582: Allow auto-commit to be set with system properties in

data_driven_schema_configs and
enable auto soft-commits for the bin/solr -e cloud example using the Config API.
(Timothy Potter)

* SOLR-7183: Fix Locale blacklisting for Minikdc based tests. (Ishan

Chattopadhyaya, hossman
via Anshum Gupta)

* SOLR-7662: Refactored response writing to consolidate the logic in one place

(Noble Paul)

* SOLR-7110: Added option to optimize JavaBinCodec to minimize string Object

creation (Noble Paul)

================== 5.1.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 8.1.10.v20130312

Upgrading from Solr 5.0

-----------------------

* SolrClient query functions now declare themselves as throwing IOException in

addition to SolrServerException, to bring them in line with the update
functions.

* SolrRequest.process() is now final. Subclasses should instead be parameterized

by their corresponding SolrResponse type, and implement createResponse()

* The signature of SolrDispatchFilter.createCoreContainer() has changed to take

(String,Properties) arguments

* Deprecated the 'lib' option added to create-requesthandler as part of SOLR-6801

in 5.0 release.
Please use the add-runtimelib command

* Tika's runtime dependency of 'jhighlight' was removed as the latter was found to
contain some LGPL-only code. Until that's resolved by Tika, you can download the
.jar yourself and place it under contrib/extraction/lib.

* The _text catch-all field in data_driven_schema_configs has been renamed to

_text_.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-6909: Extract atomic update handling logic into AtomicUpdateDocumentMerger

class
and enable subclassing. (Steve Davids, yonik)

* SOLR-6845: Add a “buildOnStartup” option for suggesters. (Tomás Fernández Löbbe)

* SOLR-6449: Add first class support for Real Time Get in Solrj.
(Anurag Sharma, Steve Davids via shalin)

* SOLR-6954: SolrClient now implements Closeable, and shutdown() has been

deprecated in favour of close(). (Mark Miller, Tomás Fernández Löbbe, Alan
Woodward)

* SOLR-4905: Allow fromIndex parameter to JoinQParserPlugin to refer to a single-

sharded
collection that has a replica on all nodes where there is a replica in the to
index
(Jack Lo, Timothy Potter)

* SOLR-6648: Add support in AnalyzingInfixLookupFactory and

BlendedInfixLookupFactory
for setting 'highlight' and 'allTermsRequired' in the suggester configuration.
(Boon Low, Varun Thacker via Tomás Fernández Löbbe)

* SOLR-7083: Support managing all named components in solrconfig such as

requestHandler, queryParser, queryResponseWriter, valueSourceParser,
transformer, queryConverter (Noble Paul)

* SOLR-7005: Spatial 2D heatmap faceting on RPT fields via new facet.heatmap with
PNG and
2D int array formats. (David Smiley)

* SOLR-7019: Support changing field key when using interval faceting.

(Tomás Fernández Löbbe)

* SOLR-6832: Queries be served locally rather than being forwarded to another

replica.
(Sachin Goyal, Timothy Potter)

* SOLR-1945: Add support for child docs in DocumentObjectBinder (Noble Paul, Mark
Miller)

* SOLR-7125, SOLR-7158: You can upload and download configurations via

CloudSolrClient
(Alan Woodward, Ishan Chattopadhyaya)

* SOLR-5507: Admin UI - Refactoring using AngularJS, first part (Upayavira via

Erick Erickson)

* SOLR-7164: BBoxField defaults sub fields to not-stored (ryan)

* SOLR-7155,SOLR-7201: All SolrClient methods now take an optional 'collection'

argument
(Alan Woodward, Shawn Heisey)

* SOLR-6359: Allow number of logs and records kept by UpdateLog to be configured

(Ramkumar Aiyengar)

* SOLR-7189: Allow DIH to extract content from embedded documents via Tika.
(Tim Allison via shalin)

* SOLR-6841: Visualize lucene segment information in Admin UI.

(Alexey Kozhemiakin, Michal Bienkowski, hossman, Shawn Heisey, Varun Thacker
via shalin)

* SOLR-5846: EnumField supports DocValues functionality. (Elran Dvir, shalin)

* SOLR-4044: CloudSolrClient.connect() throws a more useful exception if the

cluster is not ready, and can now take an optional timeout argument to wait
for the cluster. (Alan Woodward, shalin, yonik, Mark Miller, Vitaliy Zhovtyuk)

* SOLR-7073: Support adding a jar to a collections classpath (Noble Paul)

* SOLR-7126: Secure loading of runtime external jars (Noble Paul)

* SOLR-6349: Added support for stats.field localparams to enable/disable individual

stats to
limit the amount of computation done and the amount of data returned.
eg: stats.field={!min=true max=true}field_name
(Tomas Fernandez-Lobbe, Xu Zhang, hossman)

* SOLR-7218: lucene/solr query syntax to give any query clause a constant score.
General Form: <clause>^=<constant_score>
Example: (color:blue color:green)^=2.0 text:shoes
(yonik)

* SOLR-7214: New Facet module with a JSON API, facet functions, aggregations, and
analytics.
Any facet type can have sub facets, and facets can be sorted by arbitrary
aggregation functions.
Examples:
json.facet={x:'avg(price)', y:'unique(color)'}
json.facet={count1:{query:"price:[10 TO 20]"}, count2:{query:"color:blue AND
popularity:[0 TO 50]"} }
json.facet={categories:{terms:{field:cat, sort:"x desc", facet:{x:"avg(price)",
y:"sum(price)"}}}}
(yonik)

* SOLR-6141: Schema API: Remove fields, dynamic fields, field types and copy
fields; and replace fields, dynamic fields and field types. (Steve Rowe)

* SOLR-7217: HTTP POST body is auto-detected when the client is curl and the
content
type is form data (curl's default), allowing users to use curl to send
JSON or XML without having to specify the content type. (yonik)

* SOLR-6892: Update processors can now be top-level components and they can be
specified in request to create a new custom update chain (Noble Paul)

* SOLR-7216: Solr JSON Request API:

- HTTP search requests can have a JSON body.
- JSON request can also be passed via the "json" parameter.
- Smart merging of multiple JSON parameters: ruery parameters starting with
"json."
will be merged into the JSON request.
- Legacy query parameters can also be passed in the "params" block of
the JSON request.
(yonik)

* SOLR-7245: Temporary ZK election or connection loss should not stall indexing

due to leader initiated recovery (Ramkumar Aiyengar)

* SOLR-6350: StatsComponent now supports Percentiles (Xu Zhang, hossman)

* SOLR-7306: Percentiles support for the new facet module. Percentiles

can be calculated for all facet buckets and field faceting can sort
by percentile values.
Examples:
json.facet={ median_age : "percentile(age,50)" }
json.facet={ salary_percentiles : "percentile(salary,25,50,75)" }
(yonik)

* SOLR-7307: EmbeddedSolrServer can now be started up by passing a path to a

solr home directory, or a NodeConfig object (Alan Woodward, Mike Drob)

* SOLR-1387: Add facet.contains and facet.contains.ignoreCase options (Tom Winch

via Alan Woodward)

* SOLR-7082: Streaming Aggregation for SolrCloud (Joel bernstein, Yonik Seeley)

* SOLR-7212: Parameter substitution / macro expansion across entire request.

Substitution can contain further expansions and default values are supported.
Example: q=price:[ ${low:0} TO ${high} ]&low=100&high=200
(yonik)

* SOLR-7226: Make /query/* jmx/* , requestDispatcher/*, <listener> <initParams>

properties in solrconfig.xml editable (Noble Paul)

* SOLR-7240: '/' redirects to '/solr/' for convenience (Martijn Koster, hossman)

* SOLR-5911: Added payload support for term vectors. New "termPayloads" option for
fields
/ types in the schema, and "tv.payloads" param for the term vector component.
(Mike McCandless, David Smiley)

* SOLR-5132: Added a new collection action MODIFYCOLLECTION (Noble Paul)

Bug Fixes
----------------------

* SOLR-7046: NullPointerException when group.function uses query() function.

 (Jim Musil via Erick Erickson)

* SOLR-7072: Multiple mlt.fl does not work. (Constantin Mitocaru, shalin)

* SOLR-6775: Creating backup snapshot results in null pointer exception.

(Ryan Hesson, Varun Thacker via shalin)

* SOLR-5890: Delete silently fails if not sent to shard where document was
added (Ishan Chattopadhyaya, Noble Paul)

* SOLR-7101: JmxMonitoredMap can throw an exception in clear when queryNames fails.

(Mark Miller, Wolfgang Hoschek)

* SOLR-6214: Snapshots numberToKeep param only keeps n-1 backups.

(Mathias H., Ramana, Varun Thacker via shalin)

* SOLR-7084: FreeTextSuggester: Better error message when doing a lookup

during dictionary build. Used to be nullpointer (janhoy)

* SOLR-6956: OverseerCollectionProcessor and replicas on the overseer node can

sometimes
operate on stale cluster state due to overseer holding the state update lock for
a
long time. (Mark Miller, shalin)

* SOLR-7104: Propagate property prefix parameters for ADDREPLICA Collections API

call.
(Varun Thacker via Anshum Gupta)

* SOLR-7113: Multiple calls to UpdateLog#init is not thread safe with respect to

the
HDFS FileSystem client object usage. (Mark Miller, Vamsee Yarlagadda)

* SOLR-7128: Two phase distributed search is fetching extra fields in GET_TOP_IDS

phase.
(Pablo Queixalos, shalin)

* SOLR-7139: Fix SolrContentHandler for TIKA to ignore multiple startDocument

events.
(Chris A. Mattmann, Uwe Schindler)

* SOLR-7178: OverseerAutoReplicaFailoverThread compares Integer objects using ==

(shalin)

* SOLR-7171: BaseDistributedSearchTestCase now clones getSolrHome() for each

subclass,
and consistently uses getSolrXml(). (hossman)

* SOLR-6657: DocumentDictionaryFactory requires weightField to be mandatory, but

it shouldn't
(Erick Erickson)

* SOLR-7206: MiniSolrCloudCluster wasn't dealing with SSL mode correctly (Alan

Woodward)

* SOLR-4464: DIH Processed documents counter resets to zero after first entity is
processed.
(Dave Cook, Shawn Heisey, Aaron Greenspan, Thomas Champagne via shalin)
* SOLR-7209: /update/json/docs carry forward fields from previous records (Noble
Paul)

* SOLR-7195: Fixed a bug where the bin/solr shell script would incorrectly
detect another Solr process listening on the same port number. If the
requested listen port was 8983, it would match on another Solr using port
18983 for any purpose. Also escapes the dot character in all grep commands
looking for start.jar.
(Xu Zhang via Shawn Heisey)

* SOLR-6682: Fix response when using EnumField with StatsComponent

(Xu Zhang via hossman)

* SOLR-7109: Indexing threads stuck during network partition can put leader into
down state.
(Mark Miller, Anshum Gupta, Ramkumar Aiyengar, yonik, shalin)

* SOLR-7092: Stop the HDFS lease recovery retries in HdfsTransactionLog on close

and try
to avoid lease recovery on closed files. (Mark Miller)

* SOLR-7285: ActionThrottle will not pause if getNanoTime first returns 0.

(Mark Miller, Gregory Chanan)

* SOLR-7141: RecoveryStrategy: Raise time that we wait for any updates from the
leader before
they saw the recovery state to have finished. (Mark Miller)

* SOLR-7248: In legacyCloud=false mode we should check if the core was hosted on

the same node before registering it
(Varun Thacker, Noble Paul, Mark Miller)

* SOLR-7294: Migrate API fails with 'Invalid status request: notfoundretried

6times' message.
(Jessica Cheng Mallet, shalin)

* SOLR-7254: Make an invalid negative start/rows throw a HTTP 400 error (Bad
Request) instead
of causing a 500 error. (Ramkumar Aiyengar, Hrishikesh Gadre, yonik)

* SOLR-7305: BlendedInfixLookupFactory swallows root IOException when it occurs.

(Stephan Lagraulet via shalin)

* SOLR-7293: Fix bug that Solr server does not listen on IPv6 interfaces by
default.
(Uwe Schindler, Sebastian Pesman)

* SOLR-7298: Fix Collections API calls (SolrJ) to not add name parameter when not
needed.
(Shai Erera, Anshum Gupta)

* SOLR-7134: Replication can still cause index corruption. (Mark Miller, shalin,
Mike Drob)

* SOLR-7309: Make bin/solr, bin/post work when Solr installation directory contains
spaces
(Ramkumar Aiyengar, Martijn Koster)

* SOLR-6924: The config API forcefully refreshes all replicas in the collection to
ensure all are
updated (Noble Paul)

* SOLR-7266: The IgnoreCommitOptimizeUpdateProcessor blocks commit requests from

replicas needing to recover. (Jessica Cheng Mallet, Timothy Potter)

* SOLR-7299: bin\solr.cmd doesn't use jetty SSL configuration. (Steve Rowe)

* SOLR-7334: Admin UI does not show "Num Docs" and "Deleted Docs". (Erick Erickson,
Timothy Potter)

* SOLR-7338, SOLR-6583: A reloaded core will never register itself as active after
a ZK session expiration
(Mark Miller, Timothy Potter)

* SOLR-7366: Can't index example XML docs into the cloud example using bin/post due
to regression in
ManagedIndexSchema's handling of ResourceLoaderAware objects used by field types
(Steve Rowe, Timothy Potter)

* SOLR-7284: HdfsUpdateLog is using hdfs FileSystem.get without turning off the

cache.
(Mark Miller)

* SOLR-7286: Using HDFS's FileSystem.newInstance does not guarantee a new instance.

(Mark Miller)

* SOLR-7508: SolrParams.toMultiMap() does not handle arrays (Thomas Scheffler ,

Noble Paul)

Optimizations
----------------------

* SOLR-7049: Move work done by the LIST Collections API call to the Collections
Handler (Varun Thacker via Anshum Gupta).

* SOLR-7116: Distributed facet refinement requests would needlessly compute other

types
of faceting that have already been computed. (David Smiley, Hossman)

* SOLR-7239: improved performance of min & max in StatsComponent, as well as

situations
where local params disable all stats (hossman)

* SOLR-7050: realtime get should internally load only fields specified in fl.
(yonik, Noble Paul)

Other Changes
----------------------

* SOLR-7014: Collapse identical catch branches in try-catch statements. (shalin)

* SOLR-6500: Refactor FileFetcher in SnapPuller, add debug logging.

(Ramkumar Aiyengar via Mark Miller)

* SOLR-7076: In DIH, TikaEntityProcessor should have support for onError=skip

(Noble Paul)

* SOLR-7094: Better error reporting of JSON parse issues when indexing docs
 (Ishan Chattopadhyaya via Timothy Potter)

* SOLR-7103: Remove unused method params in faceting code. (shalin)

* SOLR-6311: When performing distributed queries, SearchHandler should use path

when no qt or shard.qt parameter is specified; fix also resolves SOLR-4479.
(Steve Molloy, Timothy Potter)

* SOLR-7112: Fix DeleteInactiveReplicaTest.deleteLiveReplicaTest test failures.

(shalin)

* SOLR-6902: Use JUnit rules instead of inheritance with distributed Solr

tests to allow for multiple tests without the same class.
(Ramkumar Aiyengar, Erick Erickson, Mike McCandless)

* SOLR-7032: Clean up test remnants of old-style solr.xml (Erick Erickson)

* SOLR-7145: SolrRequest is now parametrized by its response type (Alan

Woodward)

* SOLR-7142: Fix TestFaceting.testFacets. (Michal Kroliczek via shalin)

* SOLR-7156: Fix test failures due to resource leaks on windows.

(Ishan Chattopadhyaya via shalin)

* SOLR-7147: Introduce new TrackingShardHandlerFactory for monitoring what requests

are sent to shards during tests. (hossman, shalin)

* SOLR-7160: Rename ConfigSolr to NodeConfig, and decouple it from xml

representation (Alan Woodward)

* SOLR-7166: Encapsulate JettySolrRunner configuration (Alan Woodward)

* SOLR-7130: Make stale state notification work without failing the requests
(Noble Paul, shalin)

* SOLR-7151: SolrClient query methods throw IOException (Alan Woodward)

* SOLR-7179: JettySolrRunner no longer passes configuration to

SolrDispatchFilter via system properties, but instead uses a Properties
object in the servlet context (Alan Woodward)

* SOLR-6275: Improve accuracy of QTime reporting (Ramkumar Aiyengar)

* SOLR-7174: DIH should reset TikaEntityProcessor so that it is capable

of re-use (Alexandre Rafalovitch , Gary Taylor via Noble Paul)

* SOLR-6804: Untangle SnapPuller and ReplicationHandler (Ramkumar Aiyengar)

* SOLR-7180: MiniSolrCloudCluster will startup and shutdown its jetties in

parallel (Alan Woodward, Tomás Fernández Löbbe, Vamsee Yarlagadda)

* SOLR-7173: Fix ReplicationFactorTest on Windows by adding better retry

support after seeing no response exceptions. (Ishan Chattopadhyaya via Timothy
Potter)

* SOLR-7246: Speed up BasicZkTest, TestManagedResourceStorage (Ramkumar Aiyengar)

* SOLR-7258: Forbid MessageFormat.format and MessageFormat single-arg constructor.

 (shalin)

* SOLR-7162: Remove unused SolrSortField interface. (yonik, Connor Warrington via

shalin)

* SOLR-6414: Update to Hadoop 2.6.0. (Mark Miller)

* SOLR-6673: MDC based logging of collection, shard, replica, core

(Ishan Chattopadhyaya , Noble Paul)

* SOLR-7291: Test indexing on ZK disconnect with ChaosMonkey tests (Ramkumar

Aiyengar)

* SOLR-7203: Remove buggy no-op retry code in HttpSolrClient (Alan Woodward,

Mark Miller, Greg Solovyev)

* SOLR-7202: Remove deprecated string action types in Overseer and

OverseerCollectionProcessor -
"deletecollection", "createcollection", "reloadcollection", "removecollection",
"removeshard".
(Varun Thacker, shalin)

* SOLR-7290: Rename catchall _text field in data_driven_schema_configs

to _text_ (Steve Rowe)

* SOLR-7346: Stored XSS in Admin UI Schema-Browser page and Analysis page (Mei Wang
via Timothy Potter)

================== 5.0.0 ==================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release.

NOTE: Solr 5.0 only supports creating and removing SolrCloud collections through
the collections API, unlike previous versions. While not using the
collections API may still work in 5.0, it is unsupported, not recommended,
and the behavior will change in a 5.x release.

Versions of Major Components

---------------------
Apache Tika 1.7
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6
Jetty 8.1.10.v20130312

Upgrading from Solr 4.x

----------------------

* Apache Solr has no support for Lucene/Solr 3.x and earlier indexes anymore.
Be sure to run Lucene's IndexUpgrader on the previous 4.10 version if you might
still have old segments in your index. Alternatively fully optimize your index
with Solr 4.10 to make sure it consists only of one up-to-date index segment.

* The "file" attribute of infoStream in solrconfig.xml is removed. Control this

via your logging configuration (org.apache.solr.update.LoggingInfoStream)
instead.
* UniqFieldsUpdateProcessorFactory no longer supports the <lst named="fields"> init
param style that was deprecated in Solr 4.5. If you are still using this syntax,
update your configs to use <arr name="fieldName"> instead. See SOLR-4249 for more
details.

* The following legacy numeric and date field types, deprecated in Solr 4.8, are no
longer supported: BCDIntField, BCDLongField, BCDStrField, IntField, LongField,
FloatField, DoubleField, SortableIntField, SortableLongField, SortableFloatField,
SortableDoubleField, and DateField. Convert these types in your schema to the
corresponding Trie-based field type and then re-index. See SOLR-5936 for more
information.

* getAnalyzer() in IndexSchema and FieldType that was deprecated in Solr 4.9 has
been removed. Use getIndexAnalyzer() instead. See SOLR-6022 for more
information.

* The spellcheck response format has changed, affecting xml and json clients. In
particular, the "correctlySpelled" and "collations" subsections have been moved
outside
the "suggestions" subsection, and now are directly under "spellcheck".
See SOLR-3029 for more information.

* The CollectionsAPI SolrJ calls createCollection(), reloadCollection(),

deleteCollection(), requestStatus(), createShard(), splitShard(), deleteShard(),
createAlias() and deleteAlias() which were deprecated in 4.11 have been removed.
The new usage involves a builder style construction of the call.

* The OVERSEERSTATUS API returns new key names for operations such as "create"
for "createcollection", "delete" for "removecollection" and "deleteshard" for
"removeshard".

* If you have been using the /update/json/docs to index documents, SOLR-6617

introduces
backward incompatible change. the key names created are fully qualified paths of
keys .
If you need the old functionality back , please add an extra parameter f=/**
example: /update/json/docs?f=/**

* Bugs fixed in several ValueSource functions may result in different behavior in

situations where some documents do not have values for fields wrapped in other
value
sources. Users who want to preserve the previous behavior may need to wrap
fields
in the "def()" function. Example: changing "fl=sum(fieldA,fieldB)" to
"fl=sum(def(fieldA,0.0),def(fieldB,0.0))". See LUCENE-5961 for more details.

* AdminHandlers is deprecated, /admin/* are implicitly defined, /get, /replication

and
handlers are also implicitly registered (refer to SOLR-6792)

* SolrCore.reload(ConfigSet coreConfig, SolrCore prev) was deprecated in 4.10.3 and

removed in 5.0. use SolrCore.reload(ConfigSet coreConfig). See SOLR-5864.

* The "termIndexInterval" option in solrconfig.xml has been a No-Op in the default

codec
since Solr 4.0, and has been removed completely in 5.0. If you get an "Illegal
parameter
'termIndexInterval'" error when upgrading, you can safely remove this option from
your
 configs. If you have a strong need to configure this, you must explicitly
configure your
schema with a custom codec. See SOLR-6560 and for more details.

* The "checkIntegrityAtMerge" option in solrconfig.xml is now a No-Op and should be

removed
from any solrconfig.xml files -- these integrity checks are now done
automatically at a very
low level during the segment merging process. See SOLR-6834 for more details.

* SimplePostTool (post.jar) no longer defaults to collection1, making either of

core/collection
name or update URL mandatory. An existing call without an explicit update URL
needs to now
have the core/collection name passed as "-Dc=<collection/core name>" e.g.:
java -jar -Dc=<collection_name> post.jar *.xml (new call with collection name)
See SOLR-6852 for more details.

* Relative paths specified in the solr.xml coreRootDirectory parameter for core

discovery are now resolved relative to SOLR_HOME, rather than cwd. See
SOLR-6718.

* SolrServer and associated classes have been deprecated. Applications using

SolrJ should use the equivalent SolrClient classes instead.

* Spatial fields originating from Solr 4 (e.g. SpatialRecursivePrefixTreeFieldType,

BBoxField)
have the 'units' attribute deprecated, now replaced with 'distanceUnits'. If you
change it to
a unit other than 'degrees' (or if you don't specify it, which will default to
kilometers if
geo=true), then be sure to update maxDistErr as it's in those units. If you keep
units=degrees
then it should be backwards compatible but you'll get a deprecation warning on
startup. See
SOLR-6797.

* The <nrtMode> configuration in solrconfig.xml has been discontinued and should be

removed from
solrconfig.xml. Solr defaults to using NRT searchers regardless of the value in
configuration
and a warning is logged on startup if the solrconfig.xml has <nrtMode> specified.

* There was an old spatial syntax to specify a circle using Circle(x,y d=...) which
should be
replaced with simply using {!geofilt} (if you can) or BUFFER(POINT(x y),d).
Likewise a rect syntax
comprised of minX minY maxX maxY that should now be replaced with
ENVELOPE(minX, maxX, maxY, minY).

* Due to changes in the underlying commons-codec package, users of the

BeiderMorseFilterFactory
will need to rebuild their indexes after upgrading. See LUCENE-6058 for more
details.

* CachedSqlEntityProcessor has been removed, use SqlEntityProcessor with the

cacheImpl parameter.

* HttpDataSource has been removed, use URLDataSource instead.

* LegacyHTMLStripCharFilter has been removed

* CoreAdminRequest.persist() call has been removed. All changes made via

CoreAdmin are persistent.

* SpellCheckResponse.getSuggestions() and getSuggestionFrequencies() have been

removed, use getAlternatives() and getAlternativeFrequencies() instead.

* SolrQuery deprecated methods have been removed:

- setMissing() is now setFacetMissing()
- getFacetSort() is now getFacetSortString()
- setFacetSort(boolean) should instead use setFacetSort(String) with
FacetParams.FACET_SORT_COUNT or FacetParams.FACET_SORT_INDEX
- setSortField(String, ORDER) should use setSort(SortClause)
- addSortField(String, ORDER) should use addSort(SortClause)
- removeSortField(String, ORDER) should use removeSort(SortClause)
- getSortFields() should use getSorts()
- set/getQueryType() should use set/getRequestHandler()

* ClientUtil deprecated date methods have been removed, use DateUtil instead

* FacetParams.FacetDateOther has been removed, use FacetRangeOther

* ShardParams.SHARD_KEYS has been removed, use ShardParams._ROUTE_

* The 'old-style' solr.xml format is no longer supported, and cores must be

defined using core.properties files. See
https://cwiki.apache.org/confluence/display/solr/Format+of+solr.xml

Detailed Change List

----------------------

New Features
----------------------

* SOLR-6103: Added DateRangeField for indexing date ranges, especially multi-valued

ones.
Supports facet.range, DateMath, and is mostly interoperable with TrieDateField.
Based on LUCENE-5648. (David Smiley)

* SOLR-6403: TransactionLog replay status logging. (Mark Miller)

* SOLR-4580: Support for protecting content in ZooKeeper. (Per Steffensen, Mark

Miller)

* SOLR-6365: specify appends, defaults, invariants outside of the request handler.

(Noble Paul, Erik Hatcher, shalin)

* SOLR-5097: Schema API: Add REST support for adding dynamic fields to the schema.
(Steve Rowe)

* SOLR-5098: Schema API: Add REST support for adding field types to the schema.
(Timothy Potter)

* SOLR-5473 : Split clusterstate.json per collection and watch states selectively

(Noble Paul, Mark Miller, shalin, Jessica Cheng Mallet, Timothy Potter, Anshum
Gupta)
* SOLR-5474 : Support for SOLR-5473 in SolrJ (Timothy Potter, Noble Paul, Mark
Miller)

* SOLR-5810 : Support for SOLR-5473 in solr admin UI (Timothy Potter, Noble Paul)

* SOLR-6482: Add an onlyIfDown flag for DELETEREPLICA collections API command

(Erick Erickson)

* SOLR-6354: stats.field can now be used to generate stats over the numeric results
of arbitrary functions, ie: stats.field={!func}product(price,popularity)
(hossman)

* SOLR-6485: ReplicationHandler should have an option to throttle the speed of

replication (Varun Thacker, Noble Paul)

* SOLR-6543: Give HttpSolrClient the ability to send PUT requests (Gregory Chanan)

* SOLR-5986: Don't allow runaway queries from harming Solr cluster health or search
performance (Anshum Gupta, Steve Rowe, Robert Muir)

* SOLR-6565: SolrRequest support for query params (Gregory Chanan)

* SOLR-6476: Create a bulk mode for schema API (Noble Paul, Steve Rowe)

* SOLR-6512: Add a collections API call to add/delete arbitrary properties

to a specific replica. Optionally adding sliceUnique=true will remove
this property from all other replicas within a particular slice.
(Erick Erickson)

* SOLR-6513: Add a collectionsAPI call BALANCESLICEUNIQUE. Allows the even

distribution of custom replica properties across nodes making up a
collection, at most one node per slice will have the property.

* SOLR-6605: Make ShardHandlerFactory maxConnections configurable.

(Christine Poerschke via shalin)

* SOLR-6585: RequestHandlers can optionally handle sub paths as well (Noble Paul)

* SOLR-6617: /update/json/docs path will use fully qualified node names by default
(Noble Paul)

* SOLR-4715: Add CloudSolrClient constructors which accept a HttpClient instance.

(Hardik Upadhyay, Shawn Heisey, shalin)

* SOLR-5992: add "removeregex" as an atomic update operation

(Vitaliy Zhovtyuk via Erick Erickson)

* SOLR-6633: /update/json/docs path can now save the underlying json doc asa string
field
and better support added to the default example (Noble Paul)

* SOLR-6650: Add optional slow request logging at WARN level

(Jessica Cheng Mallet via Timothy Potter)

* SOLR-6655: SimplePostTool now features -Dhost, -Dport, and -Dc (for

core/collection)
properties to allow easier overriding of just the right piece of the Solr URL.
(ehatcher)
* SOLR-6248: MoreLikeThis QParser that accepts a document id and returns documents
that
have similar content. It works in standalone/cloud mode and shares logic with the
Lucene MoreLikeThis class (Anshum Gupta).

* SOLR-6670: change BALANCESLICEUNIQUE to BALANCESHARDUNIQUE. Also, the parameter

for ADDREPLICAPROP that used to be sliceUnique is now shardUnique. (Erick
Erickson)

* SOLR-6351: Stats can now be nested under pivot values by adding a 'stats' local
param to
facet.pivot which refers to a 'tag' local param in one or more stats.field
params.
(hossman, Vitaliy Zhovtyuk, Steve Molloy)

* SOLR-6533: Support editing common solrconfig.xml values (Noble Paul)

* SOLR-6607: Managing requesthandlers through API (Noble Paul)

* SOLR-4799: faster join using join="zipper" aka merge join for nested DIH
EntityProcessors
(Mikhail Khludnev via Noble Paul)

* SOLR-6787: API to manage blobs in Solr (Noble Paul)

* SOLR-6801: Load RequestHandler from blob store (Noble Paul)

* SOLR-1632: Support Distributed IDF (Andrzej Bialecki, Mark Miller, Yonik Seeley,
Robert Muir, Markus Jelsma, Vitaliy Zhovtyuk, Anshum Gupta)

* SOLR-6729: createNodeSet.shuffle=(true|false) support for /admin/collections?

action=CREATE.
(Christine Poerschke, Ramkumar Aiyengar via Mark Miller)

* SOLR-6851: Scripts to support installing and running Solr as a service on Linux

(Timothy Potter, Hossman, Steve Rowe)

* SOLR-6770: Add/edit param sets and use them in Requests (Noble Paul)

* SOLR-6879: Have an option to disable autoAddReplicas temporarily for all

collections.
(Varun Thacker via Steve Rowe)

* SOLR-6435: Add bin/post script to simplify posting content to Solr (Erik Hatcher)

* SOLR-6761: Ability to ignore commit and/or optimize requests from clients when
running in
SolrCloud mode using the IgnoreCommitOptimizeUpdateProcessorFactory. (Timothy
Potter)

* SOLR-6797: Spatial fields that used to require units=degrees like

SpatialRecursivePrefixTreeFieldType (RPT) now take distanceUnits=degrees|
kilometers|miles
instead. It is applied to nearly all distance measurements involving the field:
maxDistErr,
distErr, d, geodist, score=distance|area|area2d. score now accepts these units as
well. It does
NOT affect distances embedded in WKT strings like BUFFER(POINT(200 10),0.2)).
(Ishan Chattopadhyaya, David Smiley)
* SOLR-6766: Expose HdfsDirectoryFactory Block Cache statistics via JMX.
(Mike Drob, Mark Miller)

* SOLR-2035: Add a VelocityResponseWriter $resource tool for locale-specific string

lookups.
(Erik Hatcher)

* SOLR-6916: Toggle payload support for the default highlighter via hl.payloads.
It's auto
enabled when the index has payloads. (David Smiley)

* SOLR-6581: Efficient DocValues support and numeric collapse field implementations

for Collapse and Expand (Joel Bernstein)

* SOLR-6937: In schemaless mode ,replace spaces and special characters with

underscore (Noble Paul)

* SOLR-5147: Support child documents in DIH

(Vadim Kirilchuk, Shawn Heisey, Thomas Champagne, Mikhail Khludnev via Noble
Paul)

Bug Fixes
----------------------

* SOLR-4895: An error should be returned when a rollback is attempted in SolrCloud

mode.
(Vamsee Yarlagadda via Mark Miller)

* SOLR-6424: The hdfs block cache BLOCKCACHE_WRITE_ENABLED is not defaulting to

false like it
should. (Mark Miller)

* SOLR-6426: SolrZkClient clean can fail due to a race with children nodes. (Mark
Miller)

* SOLR-5966: Admin UI Menu is fixed and doesn't respect smaller viewports.

(Aman Tandon, steffkes via shalin)

* SOLR-4406: Fix RawResponseWriter to respect 'base' writer

(Steve Davids, hossman)

* SOLR-6297: Fix WordBreakSolrSpellChecker to not lose suggestions in shard/cloud

environments (James Dyer)

* SOLR-6467: bin/solr script should direct stdout/stderr when starting in the

background
to the solr-PORT-console.log in the logs directory instead of bin. (Timothy
Potter)

* SOLR-6187: SOLR-6154: facet.mincount ignored in range faceting using distributed

search
NOTE: This does NOT fixed for the (deprecated) facet.date idiom, use facet.range
instead. (Erick Erickson, Zaccheo Bagnati, Ronald Matamoros, Vamsee Yalargadda)

* SOLR-6457: LBHttpSolrClient: ArrayIndexOutOfBoundsException risk if counter

overflows
(longkey via Noble Paul)
* SOLR-6499: Log warning about multiple update request handlers
(Noble Paul, Andreas Hubold, hossman)

* SOLR-6507: Fixed several bugs involving stats.field used with local params
(hossman)

* SOLR-6481: CLUSTERSTATUS should check if the node hosting a replica is live when
reporting replica status (Timothy Potter)

* SOLR-6484: SolrCLI's healthcheck action needs to check live nodes as part of

reporting
the status of a replica (Timothy Potter)

* SOLR-6540 Fix NPE from strdist() func when doc value source does not exist in a
doc (hossman)

* SOLR-6624 Spelling mistakes in the Java source (Hrishikesh Gadre)

* SOLR-6307: Atomic update remove does not work for int array or date array
(Anurag Sharma , noble)

* SOLR-6224: Post soft-commit callbacks are called before soft commit actually
happens.
(shalin)

* SOLR-6591: Overseer can use stale cluster state and lose updates for collections
with stateFormat > 1. (shalin)

* SOLR-6631: DistributedQueue spinning on calling zookeeper getChildren()

(Jessica Cheng Mallet, Mark Miller, Timothy Potter)

* SOLR-6579: SnapPuller Replication blocks clean shutdown of tomcat

(Philip Black-Knight via Noble Paul)

* SOLR-6721: ZkController.ensureReplicaInLeaderInitiatedRecovery puts replica

in local map before writing to ZK. (shalin)

* SOLR-6679: Disabled suggester component from techproduct solrconfig.xml since

it caused long startup times on large indexes even when it wasn't used.
(yonik, hossman)

* SOLR-6738: Admin UI - Escape Data on Plugins-View (steffkes)

* SOLR-3774: Solr adds RequestHandler SolrInfoMBeans twice to the JMX server.

(Tomás Fernández Löbbe, hossman, Mark Miller)

* SOLR-6763: Shard leader elections should not persist across session expiry
(Alan Woodward, Mark Miller)

* SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor:

- Added langid.maxFieldValueChars and langid.maxTotalChars params to limit
input, by default 10k and 20k chars, respectively.
- Moved input concatenation to Tika implementation; the langdetect
implementation instead appends each input piece via the langdetect API.
(Vitaliy Zhovtyuk, Tomás Fernández Löbbe, Rob Tulloh, Steve Rowe)

* SOLR-6626: NPE in FieldMutatingUpdateProcessor when indexing a doc with

null field value (Noble Paul)
* SOLR-6604: SOLR-6812: Fix NPE with distrib.singlePass=true and expand
component. Increased test coverage of expand component with docValues.
(Christine Poerschke, Per Steffensen, shalin)

* SOLR-6718: Core discovery was walking paths relative to the Jetty working
directory, rather than SOLR_HOME. (Andreas Hubold, Alan Woodward)

* SOLR-6864: Support registering searcher listeners in

SolrCoreAware.inform(SolrCore)
method. Existing components rely on this. (Tomás Fernández Löbbe)

* SOLR-6850: AutoAddReplicas makes a call to wait to see live replicas that times
out after 30 milliseconds instead of 30 seconds. (Varun Thacker via Mark Miller)

* SOLR-6397: zkcli script put/putfile should allow overwriting an existing znode's

data
(Timothy Potter)

* SOLR-6873: Lib relative path is incorrect for techproduct configset

(Alexandre Rafalovitch via Erick Erickson)

* SOLR-6899: Change public setter for CollectionAdminRequest.action to protected.

(Anshum Gupta)

* SOLR-6779: fix /browse for schemaless example (ehatcher)

* SOLR-6874: There is a race around SocketProxy binding to it's port the way we
setup
JettySolrRunner and SocketProxy. (Mark Miller, Timothy Potter)

* SOLR-6735: Make CloneFieldUpdateProcessorFactory null safe (Steve Davids via

ehatcher)

* SOLR-6907: URLEncode documents directory in MorphlineMapperTest to handle spaces

etc.
in file name. (Ramkumar Aiyengar via Erick Erickson)

* SOLR-6880: Harden ZkStateReader to expect that getCollectionLive may return null

as it's contract states. (Mark Miller, shalin)

* SOLR-6643: Fix error reporting & logging of low level JVM Errors that occur when
loading/reloading a SolrCore (hossman)

* SOLR-6839: Direct routing with CloudSolrServer will ignore the Overwrite document
option.
(Mark Miller)

* SOLR-6793: ReplicationHandler does not destroy all of it's created SnapPullers.

(Mark Miller)

* SOLR-6946: Document -p port option for the create_core and create_collection

actions in
bin/solr (Timothy Potter)

* SOLR-6923: AutoAddReplicas also consults live_nodes to see if a state change has

happened.
(Varun Thacker via Anshum Gupta)

* SOLR-6941: DistributedQueue#containsTaskWithRequestId can fail with NPE. (Mark

Miller)

* SOLR-6764: Field types need to be re-informed after reloading a managed schema

from ZK
(Timothy Potter)

* SOLR-6931: We should do a limited retry when using HttpClient.

(Mark Miller, Hrishikesh Gadre, Gregory Chanan)

* SOLR-7004: Add a missing constructor for

CollectionAdminRequest.BalanceShardUnique that
sets the collection action. (Anshum Gupta)

* SOLR-6993: install_solr_service.sh won't install on RHEL / CentOS

(David Anderson via Timothy Potter)

* SOLR-6928: solr.cmd stop works only in english (john.work, Jan Høydahl, Timothy
Potter)

* SOLR-7011: Delete collection returns before collection is actually removed.

(Christine Poerschke via shalin)

* SOLR-6640: Close searchers before rollback and recovery to avoid index

corruption.
(Robert Muir, Varun Thacker, shalin)

* SOLR-6847: LeaderInitiatedRecoveryThread compares wrong replica's state with

lirState.
(shalin)

* SOLR-6856: Restore ExtractingRequestHandler's ability to capture all HTML tags

when
parsing (X)HTML. (hossman, Uwe Schindler, ehatcher, Steve Rowe)

* SOLR-7024: Improved error messages when java is not found by the bin/solr
shell script, particularly when JAVA_HOME has an invalid location.
(Shawn Heisey)

* SOLR-7038: Validate the presence of configset before trying to create a

collection.
(Anshum Gupta, Mark Miller)

* SOLR-7037: bin/solr start -e techproducts -c fails to start Solr in cloud mode

(Timothy Potter)

* SOLR-7016: Fix bin\solr.cmd to work in a directory with spaces in the name.

(Timothy Potter, Uwe Schindler)

* SOLR-6969: When opening an HDFSTransactionLog for append we must first attempt to

recover
it's lease to prevent data loss. (Mark Miller, Praneeth Varma, Colin McCabe)

* SOLR-7067: bin/solr won't run under bash 4.2+. (Steve Rowe)

* SOLR-7068: Collapse on numeric field breaks when min/max values are negative.
(Joel Bernstein)

* SOLR-6780: Fixed a bug in how default/appends/invariants params were affecting

the set
 of all "keys" found in the request parameters, resulting in some key=value param
pairs
being duplicated. This was noticeably affecting some areas of the code where
iteration
was done over the set of all params:
- literal.* in ExtractingRequestHandler
- facet.* in FacetComponent
- spellcheck.[dictionary name].* and spellcheck.collateParam.* in
SpellCheckComponent
- olap.* in AnalyticsComponent
(Alexandre Rafalovitch & hossman)

* SOLR-6920: A replicated index can end up corrupted when small files end up with
the same
file name and size. (Varun Thacker, Mark Miller)

* SOLR-7033, SOLR-5961: RecoveryStrategy should not publish any state when

closed / cancelled and there should always be a pause between recoveries
even when recoveries are rapidly stopped and started as well as when a
node attempts to become the leader for a shard.
(Mark Miller, Maxim Novikov)

* SOLR-6693: bin\solr.cmd doesn't support 32-bit JRE/JDK running on Windows due to

parenthesis in JAVA_HOME. (Timothy Potter, Christopher Hewitt, Jan Høydahl)

* SOLR-12662: Reproducing TestPolicy failures: NPE and NoClassDefFoundError.

(Steve Rowe)

Optimizations
----------------------

* SOLR-6603: LBHttpSolrClient - lazily allocate skipped-zombie-servers list.

(Christine Poerschke via shalin)

* SOLR-6554: Speed up overseer operations avoiding cluster state reads from

zookeeper at the start of each loop and instead relying on local state and
compare-and-set writes. This change also adds batching for consecutive messages
belonging to the same collection with stateFormat=2. (shalin)

* SOLR-6680: DefaultSolrHighlighter can sometimes avoid CachingTokenFilter with

hl.usePhraseHighlighter, and can be more efficient handling data from term
vectors.
(David Smiley)

* SOLR-6666: Dynamic copy fields are considering all dynamic fields, causing
a significant performance impact on indexing documents. (Liram Vardi via Erick
Erickson, Steve Rowe)

Other Changes
----------------------

* SOLR-4622: Hardcoded SolrCloud defaults for hostContext and hostPort that

were deprecated in 4.3 have been removed completely. (hossman)

* SOLR-5936: Removed deprecated non-Trie-based numeric & date field types.

(Steve Rowe)

* SOLR-6169: Finish removal of CoreAdminHandler handleAlias action begun in 4.9

(Alan Woodward)
* SOLR-6215: TrieDateField should directly extend TrieField instead of
forwarding to a wrapped TrieField. (Steve Rowe)

* SOLR-3029: Changes to spellcheck response format (Nalini Kartha via James Dyer)

* SOLR-3957: Removed RequestHandlerUtils#addExperimentalFormatWarning(), which

removes "experimental" warning from two places: replication handler details
command and DataImportHandler responses. (ehatcher)

* SOLR-6073: Remove helper methods from CollectionsRequest (SolrJ) for

CollectionsAPI
calls and move to a builder design for the same. (Varun Thacker, Anshum Gupta)

* SOLR-6519: Make DirectoryFactory#create() take LockFactory.

(Uwe Schindler)

* SOLR-6400: SolrCloud tests are not properly testing session expiration. (Mark
Miller)

* LUCENE-5650: Tests can no longer write to CWD. Update log dir is now made
relative
to the instance dir if it is not an absolute path. (Ryan Ernst, Dawid Weiss)

* SOLR-6390: Remove unnecessary checked exception for CloudSolrClient

constructors, improve javadocs for CloudSolrClient constructors.
(Steve Davids via Shawn Heisey)

* LUCENE-5901: Replaced all occurrences of LUCENE_CURRENT with LATEST for

luceneMatchVersion.
(Ryan Ernst)

* SOLR-6445: Upgrade Noggit to version 0.6 to support more flexible JSON input
(Noble Paul, Yonik Seeley)

* SOLR-6073: Remove helper methods from CollectionsRequest (SolrJ) for

CollectionsAPI
calls and move to a builder design for the same. (Varun Thacker, Anshum Gupta)

* SOLR-5322: core discovery can fail w/NPE and no explanation if a non-readable

directory exists
(Said Chavkin, Erick Erickson)

* SOLR-6488, SOLR-6991: Update to Apache Tika 1.7. This adds support for parsing
Outlook PST and
Matlab MAT files. Parsing for NetCDF files was removed because of license issues;
if you need
support for this format, download the parser JAR yourself and add it to
contrib/extraction/lib
folder: http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/
(Uwe Schindler)

* SOLR-6115: Cleanup enum/string action types in Overseer,

OverseerCollectionProcessor and
CollectionHandler. (Erick Erickson, shalin)

* SOLR-6453: Stop throwing an error message from Overseer when node exits (Ramkumar
Aiyengar, Noble Paul)
* SOLR-6550: Provide simple mechanism for passing additional metadata / context
about a server-side
SolrException back to the client-side (Timothy Potter)

* SOLR-6249: Schema API changes return success before all cores are updated; client
application
can provide the optional updateTimeoutSecs parameter to cause the server handling
the
managed schema update to block until all replicas of the same collection have
processed the
update or until the specified timeout is reached (Timothy Potter)

* SOLR-6597: SolrIndexConfig parameter in one of the SolrIndexSearcher constructor

has been removed.
It was just passed and never used via that constructor. (Anshum Gupta)

* SOLR-5852: Add CloudSolrClient helper method to connect to a ZK ensemble. (Varun

Thacker, Furkan KAMACI,
Shawn Heisey, Mark Miller, Erick Erickson via shalin)

* SOLR-6592: Avoid waiting for the leader to see the down state if that leader is
not live.
(Timothy Potter)

* SOLR-6641: SystemInfoHandler should include the zkHost the node is using (when
running in solrcloud mode)
(Timothy Potter)

* SOLR-6295: Fix child filter query creation to never match parent docs in
SolrExampleTests.
(Varun Thacker, Mikhail Khludnev via shalin)

* SOLR-6578: Update commons-io dependency to the latest 2.4 version

(Steve Rowe, Shawn Heisey)

* SOLR-6651: Fix wrong timeout logged in waitForReplicasToComeUp. (shalin)

* SOLR-6698: Solr is not consistent wrt ZkCredentialsProvider /

ZkCredentialProvider.
References to zkCredentialProvider in System properties or configurations should
be
changed to zkCredentialsProvider. (Gregory Chanan)

* SOLR-6715: ZkSolrResourceLoader constructors accept a parameter called

'collection'
but it should be 'configName'. (shalin)

* SOLR-6697: bin/solr start scripts allow setting SOLR_OPTS in solr.in.* (janhoy)

* SOLR-6739: Admin UI - Sort list of command line args (steffkes)

* SOLR-6740: Admin UI - improve Files View (steffkes)

* SOLR-6570: Run SolrZkClient session watch asynchronously.

(Ramkumar Aiyengar via Mark Miller)

* SOLR-6747: Add an optional caching option as a workaround for SOLR-6586.

(Mark Miller, Gregory Chanan)
* SOLR-6459: Normalize logging of operations in Overseer and log current queue
size.
(Ramkumar Aiyengar, shalin via Mark Miller)

* SOLR-6754: ZkController.publish doesn't use the updateLastState parameter.

(shalin)

* SOLR-6751: Exceptions thrown in the analysis chain in DirectUpdateHandler2

should return a BAD_REQUEST status (Alan Woodward)

* SOLR-6792: deprecate AdminHandlers, Clean up solrconfig.xml of

unnecessary plugin definitions, implicit registration of /replication,
/get and /admin/* handlers (Noble Paul)

* SOLR-5864: Remove previous SolrCore as parameter on reload.

(Tomás Fernández Löbbe)

* SOLR-4792: Stop shipping a .war. (Robert Muir, Ramkumar Aiyengar, Mark Miller)

* SOLR-6799: Update Saxon-HE to 9.6.0-2. (Mark Miller)

* SOLR-6454: Suppress EOFExceptions in SolrDispatchFilter.

(Ramkumar Aiyengar via Mark Miller)

* SOLR-6370: Allow tests to report/fail on many ZK watches being parallelly

requested on the same data (Ramkumar Aiyengar via Timothy Potter)

* SOLR-6752: Buffer Cache allocate/lost metrics should be exposed.

(Mike Drob via Mark Miller)

* SOLR-6560: Purge termIndexInterval from example/test configs

(Tom Burton-West, hossman)

* SOLR-6773: Remove the multicore example as the DIH and cloud examples
illustrate multicore behavior (hossman, Timothy Potter)

* SOLR-6834: Warn if checkIntegrityAtMerge is configured. This option is no longer

meaningful
since the checks are done automatically at a very low level in the segment
merging.
This warning will become an error in Solr 6.0. (hossman)

* SOLR-6833: Examples started with bin/solr -e should use a solr.solr.home

directory under
the example directory instead of server/solr. (Alexandre Rafalovitch, Anshum
Gupta, hossman,
Timothy Potter)

* SOLR-6826: fieldType capitalization is not consistent with the rest of case-

sensitive field names.
(Alexandre Rafalovitch via Erick Erickson)

* SOLR-6849: HttpSolrClient.RemoteSolrException reports the URL of the remote

host where the exception occurred. (Alan Woodward)

* SOLR-6852: SimplePostTool no longer defaults to collection1 making

core/collection/update URL
mandatory. (Anshum Gupta)
* SOLR-6861: post.sh from exampledocs directory has been removed as there no longer
is a default update
URL. (Anshum Gupta)

* SOLR-5922: Add support for adding core properties to SolrJ Collection Admin
Request calls.
(Varun Thacker via Anshum Gupta).

* SOLR-6523: Provide SolrJ support for specifying stateFormat while creating

Collections.
(Anshum Gupta)

* SOLR-6881: Add split.key support for SPLITSHARD via SolrJ (Anshum Gupta)

* SOLR-6883: CLUSTERPROP API switch case does not call break. (Varun Thacker via
shalin)

* SOLR-6882: Misspelled collection API actions in ReplicaMutator exception

messages.
(Steve Rowe via shalin)

* SOLR-6867: SolrCLI should check for existence before creating a new

core/collection,
more user-friendly error reporting (no stack trace), and the ability to pass a
directory when using bin/solr to create a core or collection (Timothy Potter)

* SOLR-6885: Add core name to RecoveryThread name. (Christine Poerschke via shalin)

* SOLR-6855: bin/solr -e dih launches, but has some path cruft issues preventing
some of the
imports don't work (Hossman, Timothy Potter)

* SOLR-3711: Truncate long strings in /browse field facets (ehatcher)

* SOLR-6876: Remove unused legacy scripts.conf (Alexandre Rafalovitch via Erick

Erickson)

* SOLR-6896: Speed up tests by dropping SolrJettyRunner thread max idle time

(Alan Woodward)

* SOLR-6448: Add SolrJ support for all current Collection API calls. (Anshum Gupta)

* Fixed a typo in various solrconfig.xml files. (sdumitriu)

* SOLR-6895: SolrServer classes are renamed to *SolrClient. The existing

classes still exist, but are deprecated. (Alan Woodward, Erik Hatcher)

* SOLR-6483: Refactor some methods in MiniSolrCloudCluster tests (Steve Davids via

Erick Erickson)

* SOLR-6906: Fix typo bug in

DistributedDebugComponentTest.testCompareWithNonDistributedRequest
(Ramkumar Aiyengar via Erick Erickson)

* SOLR-6905: Test pseudo-field retrieval in distributed search.

(Ramkumar Aiyengar via shalin)

* SOLR-6897: Nuke non-NRT mode from code and configuration. (Hossman, shalin)
* SOLR-6830: Update Woodstox to 4.4.1 and StAX to 3.1.4. (ab)

* SOLR-6918: No need to log exceptions (as warn) generated when creating MBean
stats if
the core is shutting down (Timothy Potter)

* SOLR-6932: All HttpClient ConnectionManagers and SolrJ clients should always be

shutdown
in tests and regular code. (Mark Miller)

* SOLR-1723: VelocityResponseWriter improvements (Erik Hatcher)

* SOLR-6324: Set finite default timeouts for select and update. (Ramkumar Aiyengar
via Mark Miller)

* SOLR-6952: bin/solr create action should copy configset directory instead of

reusing
an existing configset in ZooKeeper by default (Timothy Potter)

* SOLR-6933: bin/solr should provide a single "create" action that creates a core
or collection depending on whether Solr is running in standalone or cloud mode
(Timothy Potter)

* SOLR-6496: LBHttpSolrClient stops server retries after the timeAllowed threshold

is met.
(Steve Davids, Anshum Gupta)

* SOLR-6904: Removed deprecated Circle & rect syntax. See upgrading notes. (David
Smiley)

* SOLR-6943: HdfsDirectoryFactory should fall back to system props for most of it's
config
if it is not found in solrconfig.xml. (Mark Miller, Mike Drob)

* SOLR-6926: "ant example" makes no sense anymore - should be "ant server"

(Ramkumar Aiyengar, Timothy Potter)

* SOLR-6982: bin/solr and SolrCLI should support SSL-related Java System Properties
(Timothy Potter)

* SOLR-6981: Add a delete action to the bin/solr script to allow deleting of

cores /
collections (with delete collection config directory from ZK) (Timothy Potter)

* SOLR-6840: Remove support for old-style solr.xml (Erick Erickson, Alan Woodward)

* SOLR-6976: Remove classes and methods deprecated in 4.x (Alan Woodward, Noble
Paul, Chris Hostetter)

* SOLR-6521: CloudSolrClient should synchronize cache cluster state loading

( Noble Paul, Jessica Cheng Mallet)

* SOLR-7018: bin/solr stop should stop if there is only one node running or
generate
an error message prompting the user to be explicit about which of multiple nodes
to stop using the -p or -all options (Timothy Potter)

* SOLR-5918: ant clean does not remove ZooKeeper data (Varun Thacker, Steve Rowe)
* SOLR-7020: 'bin/solr start' should automatically use an SSL-enabled alternate
jetty
configuration file when in SSL mode, eliminating the need for manual jetty.xml
edits.
(Steve Rowe)

* SOLR-6227: Avoid spurious failures of ChaosMonkeySafeLeaderTest by ensuring

there's
at least one jetty to kill. (shalin)

================== 4.10.4 ==================

Bug Fixes
----------------------

* SOLR-6931: We should do a limited retry when using HttpClient.

(Mark Miller, Hrishikesh Gadre, Gregory Chanan)

* SOLR-6780: Fixed a bug in how default/appends/invariants params were affecting

the set
of all "keys" found in the request parameters, resulting in some key=value param
pairs
being duplicated. This was noticeably affecting some areas of the code where
iteration
was done over the set of all params:
- literal.* in ExtractingRequestHandler
- facet.* in FacetComponent
- spellcheck.[dictionary name].* and spellcheck.collateParam.* in
SpellCheckComponent
- olap.* in AnalyticsComponent
(Alexandre Rafalovitch & hossman)

* SOLR-6426: SolrZkClient clean can fail due to a race with children nodes. (Mark
Miller)

* SOLR-6457: LBHttpSolrClient: ArrayIndexOutOfBoundsException risk if counter

overflows
(longkey via Noble Paul)

* SOLR-6481: CLUSTERSTATUS should check if the node hosting a replica is live when
reporting replica status (Timothy Potter)

* SOLR-6631: DistributedQueue spinning on calling zookeeper getChildren()

(Jessica Cheng Mallet, Mark Miller, Timothy Potter)

* SOLR-6579: SnapPuller Replication blocks clean shutdown of tomcat

(Philip Black-Knight via Noble Paul)

* SOLR-6763: Shard leader elections should not persist across session expiry
(Alan Woodward, Mark Miller)

* SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor:

- Added langid.maxFieldValueChars and langid.maxTotalChars params to limit
input, by default 10k and 20k chars, respectively.
- Moved input concatenation to Tika implementation; the langdetect
implementation instead appends each input piece via the langdetect API.
(Vitaliy Zhovtyuk, Tomás Fernández Löbbe, Rob Tulloh, Steve Rowe)

* SOLR-6850: AutoAddReplicas makes a call to wait to see live replicas that times
 out after 30 milliseconds instead of 30 seconds. (Varun Thacker via Mark Miller)

* SOLR-6839: Direct routing with CloudSolrServer will ignore the Overwrite document
option.
(Mark Miller)

* SOLR-7139: Fix SolrContentHandler for TIKA to ignore multiple startDocument

events.
(Chris A. Mattmann, Uwe Schindler)

* SOLR-6941: DistributedQueue#containsTaskWithRequestId can fail with NPE. (Mark

Miller)

* SOLR-7011: Delete collection returns before collection is actually removed.

(Christine Poerschke via shalin)

* SOLR-6856: Restore ExtractingRequestHandler's ability to capture all HTML tags

when
parsing (X)HTML. (hossman, Uwe Schindler, ehatcher, Steve Rowe)

* SOLR-6928: solr.cmd stop works only in english (john.work, Jan Høydahl, Timothy
Potter)

* SOLR-7038: Validate the presence of configset before trying to create a

collection.
(Anshum Gupta, Mark Miller)

* SOLR-7016: Fix bin\solr.cmd to work in a directory with spaces in the name.

(Timothy Potter, Uwe Schindler)

* SOLR-6693: bin\solr.cmd doesn't support 32-bit JRE/JDK running on Windows due to

parenthesis in JAVA_HOME. (Timothy Potter, Christopher Hewitt, Jan Høydahl)

* SOLR-7067: bin/solr won't run under bash 4.2+. (Steve Rowe)

* SOLR-7033, SOLR-5961: RecoveryStrategy should not publish any state when

closed / cancelled and there should always be a pause between recoveries
even when recoveries are rapidly stopped and started as well as when a
node attempts to become the leader for a shard.
(Mark Miller, Maxim Novikov)

* SOLR-6847: LeaderInitiatedRecoveryThread compares wrong replica's state with

lirState.
(shalin)

* SOLR-7128: Two phase distributed search is fetching extra fields in GET_TOP_IDS

phase.
(Pablo Queixalos, shalin)

Other Changes
----------------------

* SOLR-7147: Introduce new TrackingShardHandlerFactory for monitoring what requests

are sent to shards during tests. (hossman, shalin)

================== 4.10.3 ==================

Bug Fixes
----------------------
* SOLR-6696: bin/solr start script should not enable autoSoftCommit by default
(janhoy)

* SOLR-6704: TrieDateField type drops schema properties in branch 4.10 (Tomás

Fernández Löbbe)

* SOLR-6085: Suggester crashes when prefixToken is longer than surface form

(janhoy)

* SOLR-6323: ReRankingQParserPlugin cleaner paging and fix bug with fuzzy, range
and other queries
that need to be re-written. (Adair Kovac, Joel Bernstein)

* SOLR-6684: Fix-up /export JSON. (Joel Bernstein)

* SOLR-6781: BBoxField didn't support dynamic fields. (David Smiley)

* SOLR-6784: BBoxField's 'score' mode should have been optional. (David Smiley)

* SOLR-6510: The collapse QParser would throw a NPE when used on a DocValues field
on
an empty segment/index. (Christine Poerschke, David Smiley)

* SOLR-2927: Solr does not unregister all mbeans upon exception in constructor
causing memory leaks. (tom liu, Sharath Babu, Cyrille Roy, shalin)

* SOLR-6685: ConcurrentModificationException in Overseer Status API. (shalin)

* SOLR-6706: /update/json/docs throws RuntimeException if a nested structure

contains a non-leaf float field (Noble Paul, shalin)

* SOLR-6610: Slow startup of new clusters because

ZkController.publishAndWaitForDownStates
always times out. (Jessica Cheng Mallet, shalin, Noble Paul)

* SOLR-6662: better validation when parsing command-line options that expect a

value
(Timothy Potter)

* SOLR-6732: Fix handling of leader-initiated recovery state was String in older

versions
and is now a JSON map, caused backwards compatibility issues when doing rolling
upgrades of
a live cluster while indexing (Timothy Potter)

* SOLR-6705: Better strategy for dealing with JVM specific options in the start
scripts; remove -XX:+AggressiveOpts and only set -XX:-UseSuperWord for Java
1.7u40
to u51. (Uwe Schindler, janhoy, hossman, Timothy Potter)

* SOLR-6726: better strategy for selecting the JMX RMI port based on SOLR_PORT in
bin/solr
script (Timothy Potter)

* SOLR-6795: distrib.singlePass returns score even though not asked for.

(Per Steffensen via shalin)

* SOLR-6796: distrib.singlePass does not return correct set of fields for multi-fl-
parameter
requests. (Per Steffensen via shalin)

* SOLR-6776: Transaction log was not flushed at the end of update requests with
softCommit
specified, which could lead to data loss if the server were killed immediately
after the
update finished. (Jeffery Yuan via yonik)

Other Changes
----------------------

* SOLR-6661: Adjust all example configurations to allow overriding error-prone

relative paths for solrconfig.xml <lib> references with solr.install.dir
system property; bin/solr scripts will set it appropriately. (ehatcher)

* SOLR-6694: Auto-detect JAVA_HOME using the Windows registry if it is not set

(janhoy, Timothy Potter)

* SOLR-6653: bin/solr script should return error code >0 when something fails
(janhoy, Timothy Potter)

* SOLR-6829: Added getter/setter for lastException in DIH's ContextImpl (ehatcher)

================== 4.10.2 ==================

Bug Fixes
----------------------

* SOLR-6509: Solr start scripts interactive mode doesn't honor -z argument (Timothy
Potter)

* SOLR-6511: Fencepost error in LeaderInitiatedRecoveryThread (Timothy Potter)

* SOLR-6530: Commits under network partitions can put any node in down state.
(Ramkumar Aiyengar, Alan Woodward, Mark Miller, shalin)

* SOLR-6573: QueryElevationComponent now works with localParams in the query

(janhoy)

* SOLR-6524: Collections left in recovery state after node restart because recovery
sleep time
increases exponentially between retries. (Mark Miller, shalin)

* SOLR-6587: Misleading exception when creating collections in SolrCloud with bad

configuration.
(Tomás Fernández Löbbe)

* SOLR-6452: StatsComponent's stat 'missing' will work on fields with

docValues=true and
indexed=false (Xu Zhang via Tomás Fernández Löbbe)

* SOLR-6646: bin/solr start script fails to detect solr on non-default port and
then after
30s tails wrong log file (janhoy)

* SOLR-6647: Bad error message when missing resource from ZK when parsing Schema
(janhoy)
* SOLR-6545: Query field list with wild card on dynamic field fails.
(Burke Webster, Xu Zhang, shalin)

Other Changes
----------------------

* SOLR-6550: Provide simple mechanism for passing additional metadata / context

about a server-side
SolrException back to the client-side (Timothy Potter)

* SOLR-6486: solr start script can have a debug flag option; use -a to set
arbitrary options
(Noble Paul, Timothy Potter)

* SOLR-6549: bin/solr script should support a -s option to set the -Dsolr.solr.home

property.
(Timothy Potter)

* SOLR-6529: Stop command in the start scripts should only stop the instance that
it had started.
(Varun Thacker, Timothy Potter)

================== 4.10.1 ==================

Bug Fixes
----------------------

* SOLR-6425: If using the new global hdfs block cache option, you can end up
reading corrupt files on file name reuse. (Mark Miller, Gregory Chanan)

* SOLR-5814: CoreContainer reports incorrect & misleading path for solrconfig.xml

when there are loading problems (Pradeep via hossman)

* SOLR-6024: Fix StatsComponent when using docValues="true" multiValued="true"

(Vitaliy Zhovtyuk & Tomas Fernandez-Lobbe via hossman)

* SOLR-6493: Fix fq exclusion via "ex" local param in multivalued stats.field

(hossman)

* SOLR-6447: bin/solr script needs to pass -DnumShards=1 for boostrapping

collection1
when starting Solr in cloud mode. (Timothy Potter)

* SOLR-6501: Binary Response Writer does not return wildcard fields.

(Mike Hugo, Constantin Mitocaru, sarowe, shalin)

Other Changes
---------------------

* SOLR-6503: Removed support for parsing netcdf files in Solr Cell because
of license issues. If you need support for this format, download the parser
JAR yourself (version 4.2) and add it to contrib/extraction/lib folder:
http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/
(Uwe Schindler)

================== 4.10.0 =================

Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this
release
Versions of Major Components
---------------------
Apache Tika 1.5 (with upgraded Apache POI 3.10.1)
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6

Upgrading from Solr 4.9

----------------------

* In Solr 3.6, all primitive field types were changed to omit norms by default when
the
schema version is 1.5 or greater (SOLR-3140), but TrieDateField's default was
mistakenly
not changed. As of Solr 4.10, TrieDateField omits norms by default (see SOLR-
6211).

* Creating a SolrCore via CoreContainer.create() no longer requires an

additional call to CoreContainer.register() to make it available to clients
(see SOLR-6170).

* CoreContainer.remove() has been removed. You should now use

CoreContainer.unload() to
delete a SolrCore (see SOLR-6232).

* solr.xml parsing has been improved to better account for the expected data types
of
various options. As part of this fix, additional error checking has also been
added to
provide errors in the event of duplicated options, or unknown option names that
may
indicate a typo. Users who have modified their solr.xml in the past and now
upgrade may
get errors on startup if they have typos or unexpected options specified in their
solr.xml
file. (See SOLR-5746 for more information.)

Detailed Change List

----------------------

New Features
----------------------

* SOLR-6196: The overseerstatus collection API instruments amILeader and ZK state

update calls.
(shalin)

* SOLR-6069: The 'clusterstatus' API should return 'roles' information. (shalin)

* SOLR-6044: The 'clusterstatus' API should return live_nodes as well. (shalin)

* SOLR-5768: Add a distrib.singlePass parameter to make EXECUTE_QUERY phase fetch

all fields
and skip GET_FIELDS. (Gregg Donovan, shalin)

* SOLR-6183: New spatial BBoxField for indexing rectangles with search support for
most predicates.
 It includes extra score relevancy modes in addition to distance:
score=overlapRatio|area|area2D.
(David Smiley, Ryan McKinley)

* SOLR-6232: You can now unload/delete cores that have failed to initialize (Alan
Woodward)

* SOLR-2245: Improvements to the MailEntityProcessor:

- Support for server-side date filtering if using GMail; requires new
dependency on the Sun Gmail Java mail extensions
- Support for using the last_index_time from the previous run as the
value for the fetchMailsSince filter.
(Peter Sturge, Timothy Potter)

* SOLR-6258: Added onRollback event handler hook to Data Import Handler (DIH).
(ehatcher)

* SOLR-6263: Add DIH handler name to variable resolver as ${dih.handlerName}.

(ehatcher)

* SOLR-6216: Better faceting for multiple intervals on DV fields (Tomas Fernandez-

Lobbe
via Erick Erickson)

* SOLR-6267: Let user override Interval Faceting key with LocalParams (Tomas
Fernandez_Lobbe
via Erick Erickson)

* SOLR-6020: Auto-generate a unique key in schema-less example if data does not

have an id field.
The UUIDUpdateProcessor was improved to not require a field name in configuration
and generate
a UUID into the unique Key field.
(Vitaliy Zhovtyuk, hossman, Steve Rowe, Erik Hatcher, shalin)

* SOLR-6294: SOLR-6437: Remove the restriction of adding json by only wrapping it

in an array in a
new path /update/json/docs (Noble Paul , hossman, Yonik Seeley, Steve Rowe)

* SOLR-6302: UpdateRequestHandlers are registered implicitly /update ,

/update/json, /update/csv , /update/json/docs (Noble Paul)

* SOLR-6318: New "terms" QParser for efficiently filtering documents by a list of

values. For
many values, it's more appropriate than a boolean query. (David Smiley)

* SOLR-6283: Add support for Interval Faceting in SolrJ. (Tomás Fernández Löbbe)

* SOLR-6304 : JsonLoader should be able to flatten an input JSON to multiple docs

(Noble Paul)

* SOLR-2894: Distributed query support for facet.pivot (Dan Cooper, Erik Hatcher,
Chris Russell,
Andrew Muldowney, Brett Lucey, Mark Miller, hossman)

* SOLR-5656: Add autoAddReplicas feature for shared file systems. (Mark Miller,
Gregory Chanan)

* SOLR-5244: Exporting Full Sorted Result Sets (Erik Hatcher, Joel Bernstein)
* SOLR-3617: bin/solr and bin/solr.cmd scripts for starting, stopping, and running
Solr examples
(Timothy Potter)

* SOLR-6233: Provide basic command line tools for checking Solr status and health.
(Timothy Potter)

Bug Fixes
----------------------

* SOLR-6095 : SolrCloud cluster can end up without an overseer with overseer roles
(Noble Paul, Shalin Mangar)

* SOLR-6165: DataImportHandler should write BigInteger and BigDecimal values as

strings.
(Anand Sengamalai via shalin)

* SOLR-6189: Avoid publishing the state as down if the node is not live when
determining
if a replica should be in leader-initiated recovery. (Timothy Potter)

* SOLR-6197: The MIGRATE collection API doesn't work when legacyCloud=false is set
in cluster properties. (shalin)

* SOLR-6206: The migrate collection API fails on retry if temp collection already
exists.
(shalin)

* SOLR-6072: The 'deletereplica' API should remove the data and instance directory
by default.
(shalin)

* SOLR-6211: TrieDateField doesn't default to omitNorms=true. (Michael Ryan, Steve

Rowe)

* SOLR-6159: A ZooKeeper session expiry during setup can keep LeaderElector from
joining elections.
(Steven Bower, shalin)

* SOLR-6223: SearchComponents may throw NPE when using shards.tolerant and there is
a failure
in the 'GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG' phase. (Tomás Fernández Löbbe via
shalin)

* SOLR-6180: Callers of ManagedIndexSchema mutators should hold the

schemaUpdateLock.
(Gregory Chanan via Steve Rowe)

* SOLR-6229: Make SuggestComponent return 400 instead of 500 for bad dictionary
selected in request.
(Tomás Fernández Löbbe via shalin)

* SOLR-6235: Leader initiated recovery should use coreNodeName instead of coreName

to avoid marking
all replicas having common core name as down. (shalin)

* SOLR-6208: JettySolrRunner QueuedThreadPool's configuration code is never

executed. (dweiss via shalin)

* SOLR-6245: Socket and Connection configuration are ignored in HttpSolrServer when

passing in HttpClient.
(Patanachai Tangchaisin, shalin)

* SOLR-6137: Schemaless concurrency improvements:

- Fixed an NPE when reloading a managed schema with no dynamic copy fields
- Moved parsing and schema fields addition to after the distributed phase
- AddSchemaFieldsUpdateProcessor now uses a fixed schema rather than always
retrieving the latest, and holds the schema update lock through the entire
schema swap-out process
(Gregory Chanan via Steve Rowe)

* SOLR-6136: ConcurrentUpdateSolrServer includes a Spin Lock (Brandon Chapman,

Timothy Potter)

* SOLR-6257: More than two "!"-s in a doc ID throws an

ArrayIndexOutOfBoundsException when using the composite id router.
(Steve Rowe)

* SOLR-5746: Bugs in solr.xml parsing have been fixed to more correctly deal with
the various
datatypes of options people can specify, additional error handling of
duplicated/unidentified
options has also been added. (Maciej Zasada, hossman)

* SOLR-5847: Fixed data import abort button in admin UI. (ehatcher)

* SOLR-6264: Distributed commit and optimize are executed serially across all
replicas. (Mark Miller, Timothy Potter)

* SOLR-6163: Correctly decode special characters in managed stopwords and synonym

endpoints.
(Vitaliy Zhovtyuk, Timo Schmidt via Timothy Potter)

* SOLR-6336: DistributedQueue can easily create too many ZooKeeper Watches.

(Ramkumar Aiyengar via Mark Miller)

* SOLR-6347: DELETEREPLICA throws a NPE while removing the last Replica in a Custom
sharded collection. (Anshum Gupta)

* SOLR-6062: Fix undesirable edismax query parser effect (introduced in SOLR-2058)

in how phrase queries
generated from pf, pf2, and pf3 are merged into the main query. (Michael
Dodsworth via ehatcher)

* SOLR-6372: HdfsDirectoryFactory should use supplied Configuration for

communicating with secure kerberos.
(Gregory Chanan via Mark Miller)

* SOLR-6284: Fix NPE in OCP when non-existent sliceId is used for a

deleteShard request (Ramkumar Aiyengar via Anshum Gupta)

* SOLR-6380: Added missing context info to log message if IOException occurs in

processing tlog
(Steven Bower via hossman)

* SOLR-6383: RegexTransformer returns no results after replaceAll if regex does not

match a value.
(Alexander Kingson, shalin)

* SOLR-6387: Add better error messages throughout Solr and supply a work around for
Java bug #8047340 to SystemInfoHandler: On Turkish default locale, some JVMs fail
to fork on MacOSX, BSD, AIX, and Solaris platforms. (hossman, Uwe Schindler)

* SOLR-6338: coreRootDirectory requires trailing slash, or SolrCloud cores are

created in wrong location.
(Primož Skale via Erick Erickson)

* SOLR-6314: Facet counts duplicated in the response if specified more than once on
the request.
(Vamsee Yarlagadda, Erick Erickson)

* SOLR-6378: Fixed example/example-DIH/ issues with "tika" and "solr"

configurations, and tidied up README.txt
(Daniel Shchyokin via ehatcher)

* SOLR-6393: TransactionLog replay performance on HDFS is very poor. (Mark Miller)

* SOLR-6268: HdfsUpdateLog has a race condition that can expose a closed HDFS
FileSystem instance and should
close its FileSystem instance if either inherited close method is called. (Mark
Miller)

* SOLR-6089: When using the HDFS block cache, when a file is deleted, its
underlying data entries in the
block cache are not removed, which is a problem with the global block cache
option.
(Mark Miller, Patrick Hunt)

* SOLR-6402: OverseerCollectionProcessor should not exit for ZooKeeper

ConnectionLoss.
(Jessica Cheng via Mark Miller)

* SOLR-6405: ZooKeeper calls can easily not be retried enough on ConnectionLoss.

(Jessica Cheng, Mark Miller)

* SOLR-6410: Ensure all Lookup instances are closed via CloseHook

(hossman, Areek Zillur, Ryan Ernst, Dawid Weiss)

Optimizations
---------------------

* LUCENE-5803: Solr's schema now uses DelegatingAnalyzerWrapper. This uses less

heap
for cached TokenStreamComponents because it caches per FieldType not per Field,
so
indexes with many fields of same type just use one TokenStream per thread.
(Shay Banon, Uwe Schindler, Robert Muir)

* SOLR-6259: Reduce CPU usage by avoiding repeated costly calls to

Document.getField inside
DocumentBuilder.toDocument for use-cases with large number of fields and
copyFields.
(Steven Bower via shalin)

* SOLR-5968: BinaryResponseWriter fetches unnecessary stored fields when only

pseudo-fields
are requested. (Gregg Donovan via shalin)

* SOLR-6261: Run ZooKeeper watch event callbacks in parallel to the ZooKeeper

event thread. (Ramkumar Aiyengar via Mark Miller)

Other Changes
---------------------

* SOLR-6173: Fixed wrong failure message in TestDistributedSearch. (shalin)

* SOLR-5902: Corecontainer level mbeans are not exposed (noble)

* SOLR-6194: Allow access to DataImporter and DIHConfiguration from

DataImportHandler.
(Aaron LaBella via shalin)

* SOLR-6170: CoreContainer.preRegisterInZk() and CoreContainer.register() commands

are merged into CoreContainer.create(). (Alan Woodward)

* SOLR-6171: Remove unused SolrCores coreNameToOrig map (Alan Woodward)

* SOLR-5596: Set system property zookeeper.forceSync=no for Solr test cases.

(shalin)

* SOLR-2853: Add a unit test for the case when "spellcheck.maxCollationTries=0"

(James Dyer)

* SOLR-6240: Removed unused coreName parameter in ZkStateReader.getReplicaProps.

(shalin)

* SOLR-6241: Harden the HttpPartitionTest. (shalin)

* SOLR-6228: Fixed bug in TestReplicationHandler.doTestIndexAndConfigReplication.

(shalin)

* SOLR-6120: On Windows, when the war is not extracted, the zkcli.bat script
will print a helpful message indicating that the war must be unzipped instead
of a java error about a missing class. (shalin, Shawn Heisey)

* SOLR-6179: Better strategy for handling empty managed data to avoid spurious
warning messages in the logs. (Timothy Potter)

* SOLR-6232: CoreContainer.remove() replaced with CoreContainer.unload(). A call

to
unload will also close the core.

* SOLR-3893: DIH should not depend on mail.jar,activation.jar (Timothy Potter,

Steve Rowe)

* SOLR-6252: A couple of small improvements to UnInvertedField class.

(Vamsee Yarlagadda, Gregory Chanan, Mark Miller)

* SOLR-3345: BaseDistributedSearchTestCase should always ignore QTime.

(Vamsee Yarlagadda, Benson Margulies via Mark Miller)

* SOLR-6270: Increased timeouts for MultiThreadedOCPTest. (shalin)

* SOLR-6274: UpdateShardHandler should log the params used to configure its

 HttpClient. (Ramkumar Aiyengar via Mark Miller)

* SOLR-6194: Opened up "public" access to DataSource, DocBuilder, and

EntityProcessorWrapper
in DIH. (Aaron LaBella via ehatcher)

* SOLR-6269: Renamed "rollback" to "error" in DIH internals, including renaming

onRollback
to onError introduced in SOLR-6258. (ehatcher)

* SOLR-3622: When using DIH in SolrCloud-mode, rollback will no longer be called

when
an error occurs. (ehatcher)

* SOLR-6231: Increased timeouts and hardened the RollingRestartTest. (Noble Paul,

shalin)

* SOLR-6290: Harden and speed up CollectionsAPIAsyncDistributedZkTest. (Mark

Miller, shalin)

* SOLR-6281: Made PostingsSolrHighlighter more configurable via subclass extension.

(David Smiley)

* SOLR-6309: Increase timeouts for AsyncMigrateRouteKeyTest. (shalin)

* SOLR-2168: Added support for facet.missing in /browse field and pivot faceting.
(ehatcher)

* SOLR-4702: Added support for multiple spellcheck collations to /browse UI.

(ehatcher)

* SOLR-5664: Added support for multi-valued field highlighting in /browse UI.

(ehatcher)

* SOLR-6313: Improve SolrCloud cloud-dev scripts. (Mark Miller, Vamsee Yarlagadda)

* SOLR-6360: Remove bogus "Content-Charset" header in HttpSolrServer. (Michael

Ryan,
Uwe Schindler)

* SOLR-6362: Fix bug in TestSqlEntityProcessorDelta. (James Dyer)

* SOLR-6388: Force upgrade of Apache POI dependency in Solr Cell to version

3.10.1 to fix CVE-2014-3529 and CVE-2014-3574. (Uwe Schindler)

* SOLR-6391: Improve message for CREATECOLLECTION failure due to missing

numShards (Anshum Gupta)

================== 4.9.1 ==================

Versions of Major Components

---------------------
Apache Tika 1.5 (with upgraded Apache POI 3.10.1)
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6

Detailed Change List

----------------------

Other Changes
---------------------

* SOLR-6503: Removed support for parsing netcdf files in Solr Cell because
of license issues. If you need support for this format, download the parser
JAR yourself (version 4.2) and add it to contrib/extraction/lib folder:
http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/
(Uwe Schindler)

* SOLR-6388: Force upgrade of Apache POI dependency in Solr Cell to version

3.10.1 to fix CVE-2014-3529 and CVE-2014-3574. (Uwe Schindler)

================== 4.9.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.5
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6

Upgrading from Solr 4.8

----------------------

* Support for DiskDocValuesFormat (ie: fieldTypes configured with

docValuesFormat="Disk")
has been removed due to poor performance. If you have an existing fieldTypes
using
DiskDocValuesFormat please modify your schema.xml to remove the 'docValuesFormat'
attribute, and optimize your index to rewrite it into the default codec, prior to
upgrading to 4.9. See LUCENE-5761 for more details.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-5999: Add checkIntegrityAtMerge support to solrconfig.xml.

(Varun Thacker via Ryan Ernst)

* SOLR-6043: Add ability to set http headers in solr response

(Tomás Fernández Löbbe via Ryan Ernst)

* SOLR-5973: Pluggable Ranking Collectors and Merge Strategies

(Joel Bernstein)

* SOLR-6108: Add support for 'addreplica' Collection API in SolrJ. (shalin)

* SOLR-5468: Allow a client application to request the minium achieved

replication factor for an update request (single or batch) by sending
an optional parameter "min_rf". (Timothy Potter)

* SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin

(Joel Bernstein)
* SOLR-5285: Added a new [child ...] DocTransformer for optionally including
Block-Join descendant documents inline in the results of a search. This works
independent of whether the search itself is a block-join related query and is
supported by he xml, json, and javabin response formats.
(Varun Thacker via hossman)

* SOLR-6150: Add new AnalyticsQuery to support pluggable analytics

(Joel Bernstein)

* SOLR-6125: Allow SolrIndexWriter to close without waiting for merges

(Christine Poerschke via Alan Woodward)

* SOLR-6064: DebugComponent track output should be returned as a JSON

object rather than a list (Christine Poerschke, Alan Woodward)

Bug Fixes
----------------------

* SOLR-5956: Use coreDescriptor.getInstanceDir() instead of getRawInstanceDir()

in the SnapShooter to avoid problems when solr.solr.home is a symbolic link.
(Timothy Potter)

* SOLR-6002: Fix a couple of ugly issues around SolrIndexWriter close and

rollback as well as how SolrIndexWriter manages its ref counted directory
instance. (Mark Miller, Gregory Chanan)

* SOLR-6015: Better way to handle managed synonyms when ignoreCase=true

(Timothy Potter)

* SOLR-6104: The 'addreplica' Collection API does not support 'async' parameter.
(shalin)

* SOLR-6101: Shard splitting doesn't work when legacyCloud=false is set in

cluster properties. (shalin)

* SOLR-6111: The 'deleteshard' collection API should be able to delete a shard

in 'construction' state. (shalin)

* SOLR-6118: 'expand.sort' didn't support function queries. (David Smiley)

* SOLR-6120: zkcli.sh should expand solr.war automatically instead of throwing

ClassNotFoundException. (sebastian badea, shalin)

* SOLR-6149: Specifying the query value without any index value does not work in
Analysis browser. (Aman Tandon, shalin)

* SOLR-6145: Fix Schema API optimistic concurrency by moving it out of

ManagedIndexSchema.add(Copy)Fields() into the consumers of those methods:
CopyFieldCollectionResource, FieldCollectionResource, FieldResource,
and AddSchemaFieldsUpdateProcessorFactory.
(Gregory Chanan, Alexey Serba, Steve Rowe)

* SOLR-6146: Incorrect configuration such as wrong chroot in zk server address can

cause CloudSolrServer to leak resources. (Jessica Cheng, Varun Thacker, shalin)

* SOLR-6158: Relative configSetBase directories were resolved relative to the

container CWD, rather than solr.home. (Simon Endele, Alan Woodward)
* SOLR-5426: Fixed a bug in ReverseWildCardFilter that could cause
InvalidTokenOffsetsException when highlighting. (Uwe Schindler, Arun Kumar, via
hossman)

* SOLR-6175: DebugComponent throws NPE on shard exceptions when using

shards.tolerant.
(Tomás Fernández Löbbe via shalin)

* SOLR-6129: DateFormatTransformer doesn't resolve dateTimeFormat. (Aaron LaBella

via shalin)

* SOLR-6164: Copy Fields Schema additions are not distributed to other nodes.
(Gregory Chanan via Steve Rowe)

* SOLR-6160: An error was sometimes possible if a distributed search included

grouping
with group.facet, faceting on facet.field and either facet.range or facet.query.
(David Smiley)

* SOLR-6182: Data stored by the RestManager could not be reloaded after core
restart, causing
the core to fail to load; cast the data loaded from storage to the correct data
type.
(Timothy Potter)

Other Changes
---------------------

* SOLR-5980: AbstractFullDistribZkTestBase#compareResults always returns false

for shouldFail. (Mark Miller, Gregory Chanan)

* SOLR-5987: Add "collection" to UpdateParams. (Mark Miller, Greg Solovyev)

* SOLR-3862: Add remove" as update option for atomically removing a value

from a multivalued field (Jim Musli, Steven Bower, Alaknantha via Erick Erickson)

* SOLR-5974: Remove ShardDoc.score and use parent's ScoreDoc.score.

(Tomás Fernández Löbbe via Ryan Ernst)

* SOLR-6025: Replace mentions of CommonsHttpSolrServer with HttpSolrServer and

StreamingUpdateSolrServer with ConcurrentUpdateSolrServer. (Ahmet Arslan via
shalin)

* SOLR-6013: Fix method visibility of Evaluator, refactor DateFormatEvaluator for

extensibility. (Aaron LaBella via shalin)

* SOLR-6022: Deprecate getAnalyzer() in IndexField and FieldType, and add

getIndexAnalyzer().
(Ryan Ernst)

* SOLR-3671: Fix DIHWriter interface usage so users may implement writers that
output
documents to a location external to Solr (ex. a NoSql db). (Roman Chyla via James
Dyer)

* SOLR-5340: Add support for named snapshots (Varun Thacker via Noble Paul)

* SOLR-5495: Recovery strategy for leader partitioned from replica case. Hardening
recovery scenarios after the leader receives an error trying to forward an
 update request to a replica. (Timothy Potter)

* SOLR-6116: Refactor DocRouter.getDocRouter to accept routerName as a String.

(shalin)

* SOLR-6026: REQUESTSTATUS Collection API now also checks for submitted tasks which
are
yet to begin execution.

* SOLR-6067: Refactor duplicate Collector code in SolrIndexSearcher

(Christine Poerschke via hossman)

* SOLR-5940: post.jar reports back detailed error in case of error responses.

(Sameer Maggon, shalin, Uwe Schindler)

* SOLR-6161: SolrDispatchFilter should throw java.lang.Error back even if wrapped

in
another exception. (Miklos Christine via shalin)

* SOLR-6153: ReplicationHandler backup response format should contain backup name.

(Varun Thacker via shalin)

* SOLR-6169: Remove broken handleAlias action in CoreAdminHandler (Alan

Woodward)

* SOLR-6128: Removed deprecated analysis factories and fieldTypes from the example
schema.xml (hossman)

* SOLR-5868: HttpClient should be configured to use ALLOW_ALL_HOSTNAME hostname

verifier to simplify SSL setup. (Steve Davids via Mark Miller)

Optimizations
----------------------

* SOLR-5681: Make the processing of Collection API calls multi-threaded.

(Anshum Gupta, shalin, Noble Paul)

Build
---------------------

* SOLR-6006: Separate test and compile scope dependencies in the Solrj and
Solr contrib ivy.xml files, so that the derived Maven dependencies get
filled out properly in the corresponding POMs. (Steven Scott, Steve Rowe)

* SOLR-6130: Added com.uwyn:jhighlight dependency to, and removed asm:asm

dependency from the extraction contrib - dependencies weren't fully
upgraded with the Tika 1.4->1.5 upgrade (SOLR-5763). (Steve Rowe)

================== 4.8.1 ==================

Bug Fixes
----------------------

* SOLR-5904: ElectionContext can cancel an election when it should not if there

was an exception while trying to register as the leader.
(Mark Miller, Alan Woodward)

* SOLR-5993: ZkController can warn about shard leader conflict even after the
conflict
is resolved. (Gregory Chanan via shalin)

* SOLR-6017: Fix SimpleQParser to use query analyzer

(Ryan Ernst)

* SOLR-6029: CollapsingQParserPlugin throws ArrayIndexOutOfBoundsException

if elevated doc has been deleted from a segment. (Greg Harris, Joel Bernstein)

* SOLR-6030: Use System.nanoTime() instead of currentTimeInMills() in

LRUCache.warm.
(Tomás Fernández Löbbe via shalin)

* SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent

(Brett Lucey, hossman)

* SOLR-6023: FieldAnalysisRequestHandler throws NPE if no parameters are supplied.

(shalin)

* SOLR-5090: SpellCheckComponent sometimes throws NPE if

"spellcheck.alternativeTermCount" is set to zero (James Dyer).

* SOLR-6039: fixed debug output when no results in response

(Tomás Fernández Löbbe, hossman)

* SOLR-6035: CloudSolrServer directUpdate routing should use getCoreUrl.

(Marvin Justice, Joel Bernstein)

================== 4.8.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.5
Carrot2 3.9.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.6

Upgrading from Solr 4.7

----------------------

* In previous versions of Solr, Terms that exceeded Lucene's MAX_TERM_LENGTH were

silently ignored when indexing documents. Beginning with Solr 4.8, a document
an error will be generated when attempting to index a document with a term
that is too large. If you wish to continue to have large terms ignored,
use "solr.LengthFilterFactory" in all of your Analyzers. See LUCENE-5472 for
more details.

* Solr 4.8 requires Java 7 or greater, Java 8 is verified to be

compatible and may bring some performance improvements. When using
Oracle Java 7 or OpenJDK 7, be sure to not use the GA build 147 or
update versions u40, u45 and u51! We recommend using u55 or later.
An overview of known JVM bugs can be found on
https://cwiki.apache.org/confluence/display/lucene/JavaBugs

* ZooKeeper is upgraded from 3.4.5 to 3.4.6.

* <fields> and <types> tags have been deprecated. There is no longer any reason to
keep them in the schema file, they may be safely removed. This allows intermixing
of
<fieldType>, <field> and <copyField> definitions if desired. Currently, these
tags
are supported so either style may be implemented. TBD is whether they'll be
deprecated formally for 5.0

Detailed Change List

----------------------

System Requirements
----------------------

* LUCENE-4747, LUCENE-5514: Move to Java 7 as minimum Java version.

(Robert Muir, Uwe Schindler)

New Features
----------------------

* SOLR-5130: Implement addReplica Collections API (Noble Paul)

* SOLR-5183: JSON updates now support nested child documents using a

"_childDocument_" object key. (Varun Thacker, hossman)

* SOLR-5714: You can now use one pool of memory for for the HDFS block cache
that all collections share. (Mark Miller, Gregory Chanan)

* SOLR-5720: Add ExpandComponent to expand results collapsed by the

CollapsingQParserPlugin. (Joel Bernstein)

* SOLR-3177: Enable tagging and excluding filters in StatsComponent via the

localParams syntax. (Mathias H., Nikolai Luthman, Vitaliy Zhovtyuk, shalin)

* SOLR-1604: Wildcards, ORs etc inside Phrase Queries. (Ahmet Arslan via Erick
Erickson)

* SOLR-5477: Async execution of OverseerCollectionProcessor(CollectionsAPI)

tasks. (Anshum Gupta)

* SOLR-5865: Provide a MiniSolrCloudCluster to enable easier testing.

(Greg Chanan via Mark Miller)

* SOLR-5860: Use leaderConflictResolveWait in WaitForState during recovery/startup,

improve logging and force refresh cluster state every 15 seconds.
(Timothy Potter via shalin)

* SOLR-5749: A new Overseer status collection API exposes overseer queue sizes,
timing
statistics, success and error counts and last N failures per operation. (shalin)

* SOLR-5858: Add a hl.qparser parameter to allow you to define a queryparser

for hl.q highlight queries. If no queryparser is defined, Solr will use
the overall query's defType. (Alan Woodward)

* SOLR-4478: Allow cores to use configuration from a configsets directory

outside their instance directory. (Alan Woodward, Erick Erickson)

* SOLR-5466: A new List collections and cluster status API which clients can use
to read collection and shard information instead of reading data directly from
ZooKeeper.
(Dave Seltzer, Varun Thacker, Vitaliy Zhovtyuk, Erick Erickson, shalin)

* SOLR-5795: New DocExpirationUpdateProcessorFactory supports computing an

expiration
date for documents from the "TTL" expression, as well as automatically deleting
expired
documents on a periodic basis. (hossman)

* SOLR-5829: Allow ExpandComponent to accept query and filter query parameters

(Joel Bernstein)

* SOLR-5653: Create a RestManager to provide REST API endpoints for

reconfigurable plugins. (Tim Potter, Steve Rowe)

* SOLR-5655: Create a stopword filter factory that is (re)configurable,

and capable of reporting its configuration, via REST API.
(Tim Potter via Steve Rowe)

* SOLR-5654: Create a synonym filter factory that is (re)configurable, and

capable of reporting its configuration, via REST API.
(Tim Potter via Steve Rowe)

* SOLR-5960: Add support for basic authentication in post.jar tool, e.g.:

java -Durl="http://username:password@hostname:8983/solr/update" -jar post.jar
sample.xml
(Sameer Maggon via Uwe Schindler)

* SOLR-4864: RegexReplaceProcessorFactory should support pattern capture group

substitution in replacement string.
(Sunil Srinivasan, Jack Krupansky via Steve Rowe)

Bug Fixes
----------------------

* SOLR-5858, SOLR-4812: edismax and dismax query parsers can be used for parsing
highlight queries. (Alan Woodward, Tien Nguyen Manh)

* SOLR-5893: On restarting overseer designate , move itself to front of the queue

(Noble Paul)

* SOLR-5915: Attempts to specify the parserImpl for

solr.PreAnalyzedField fieldtype failed. (Mike McCandless)

* SOLR-5943: SolrCmdDistributor does not distribute the openSearcher parameter.

(ludovic Boutros via shalin)

* SOLR-5954: Slower DataImportHandler process caused by not reusing jdbc

connections. (Mark Miller, Paco Garcia, Raja Nagendra Kumar)

* SOLR-5897: Upgraded to jQuery 1.7.2, Solr was previously using 1.4.3, the file
was
mistakenly named 1.7.2 (steffkes)

Optimizations
----------------------
* SOLR-1880: Distributed Search skips GET_FIELDS stage if EXECUTE_QUERY
stage gets all fields. Requests with fl=id or fl=id,score are now single-pass.
(Shawn Smith, Vitaliy Zhovtyuk, shalin)
* SOLR-5783: Requests to open a new searcher will now reuse the current registered
searcher (w/o additional warming) if possible in situations where the underlying
index has not changed. This reduces overhead in situations such as deletes that
do not modify the index, and/or redundant commits. (hossman)

* SOLR-5884: When recovery is cancelled, any call to the leader to wait to see
the replica in the right state for recovery should be aborted. (Mark Miller)

Other Changes
---------------------

* SOLR-5909: Upgrade Carrot2 clustering dependency to 3.9.0. (Dawid Weiss)

* SOLR-5764: Fix recently added tests to not use absolute paths to load test-files,
use SolrTestCaseJ4.getFile() and getResource() instead; fix morphlines/map-reduce
to not duplicate test resources and fix dependencies among them.
(Uwe Schindler)

* SOLR-5765: Update to SLF4J 1.7.6. (Mark Miller)

* SOLR-5609: If legacy mode is disabled don't let cores create

slices/replicas/collections .
All operations should be performed through collection API (Noble Paul)

* SOLR-5613: Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance.

(Thomas Champagne, Shawn Heisey via shalin)

* SOLR-5771: Add SolrTestCaseJ4.SuppressSSL annotation to disable SSL (instead of

static boolean).
(Robert Muir)

* SOLR-5799: When registering as the leader, if an existing ephemeral

registration exists, wait a short time to see if it goes away.
(Mark Miller)

* LUCENE-5472: IndexWriter.addDocument will now throw an IllegalArgumentException

if a Term to be indexed exceeds IndexWriter.MAX_TERM_LENGTH. To recreate
previous
behavior of silently ignoring these terms, use LengthFilter in your Analyzer.
(hossman, Mike McCandless, Varun Thacker)

* SOLR-5825: Separate http request creating and execution in SolrJ

(Steven Bower via Erick Erickson)

* SOLR-5837: Add hashCode/equals to SolrDocument, SolrInputDocument

and SolrInputField for testing purposes. (Varun Thacker, Noble Paul,
Mark Miller)

* SOLR-5853: The createCollection methods in the test framework now reports

result of operation in the returned CollectionAdminResponse (janhoy)

* SOLR-5838: Relative SolrHome Path Bug At AbstractFullDistribZkTestBase.

(Furkan KAMACI via shalin)

* SOLR-5763: Upgrade to Tika 1.5 (Vitaliy Zhovtyuk via Steve Rowe)

* SOLR-5881: Upgrade ZooKeeper to 3.4.6 (Shawn Heisey)

* SOLR-5883: Many tests do not shutdown SolrServer.
(Tomás Fernández Löbbe via Mark Miller)

* SOLR-5898: Update to latest Kite Morphlines release: Version 0.12.1.

(Mark Miller)

* SOLR-5228: Don't require <field> or <dynamicField> be inside of <fields> -- or

that <fieldType> be inside of <types>. (Erick Erickson)

* SOLR-5903: SolrCore implements Closeable, cut over to using try-with-resources

where possible. (Alan Woodward)

* SOLR-5914: Cleanup and fix Solr's test cleanup code.

(Mark Miller, Uwe Schindler)

* SOLR-5936: Deprecate non-Trie-based numeric & date field types. (Steve Rowe)

* SOLR-5934: LBHttpSolrServer exception handling improvement and small test

improvements. (Gregory Chanan via Mark Miller)

* SOLR-5773: CollapsingQParserPlugin should make elevated documents the

group head. (David Boychuck, Joel Bernstein)

* SOLR-5937: Modernize the DIH example config sets. (Steve Rowe)

================== 4.7.2 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.8.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Detailed Change List

----------------------

Bug Fixes
----------------------

* SOLR-5951: Fixed SolrDispatchFilter to throw useful exception on startup if

SLF4j logging jars are missing. (Uwe Schindler, Hossman, Shawn Heisey)

* SOLR-5950: Maven config: make the org.slf4j:slf4j-api dependency transitive

(i.e., not optional) in all modules in which it's a dependency, including
solrj, except for the WAR, where it will remain optional.
(Uwe Schindler, Steve Rowe)

================== 4.7.1 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.8.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5
Detailed Change List
----------------------

Bug Fixes
----------------------

* SOLR-5647: The lib paths in example-schemaless will now load correctly.

(Paul Westin via Shawn Heisey)

* SOLR-5770: All attempts to match a SolrCore with its state in clusterstate.json

should be done with the CoreNodeName. (Steve Davids via Mark Miller)

* SOLR-5875: QueryComponent.mergeIds() unmarshals all docs' sort field values once

per doc instead of once per shard.
(Alexey Serba, hoss, Martin de Vries via Steve Rowe)

* SOLR-5800: Admin UI - Analysis form doesn't render results correctly when a

CharFilter is used. (steffkes)

* SOLR-5870: Admin UI - Reload on Core Admin doesn't show errors (steffkes)

* SOLR-5867: OverseerCollectionProcessor isn't properly generating https urls in

some
cases. (Steve Davids via shalin)

* SOLR-5866: UpdateShardHandler needs to use the system default scheme registry to

properly handle https via javax.net.ssl.* properties. (Steve Davids via shalin)

* SOLR-5782: The full MapReduceIndexer help text does not display when using --
help.
(Mark Miller, Wolfgang Hoschek)

* SOLR-5824: Merge up Solr MapReduce contrib code to latest external changes.

Includes a few minor bug fixes.
(Mark Miller)

* SOLR-5818: distrib search with custom comparator does not quite work correctly
(Ryan Ernst)

* SOLR-5895: JavaBinLoader hides IOExceptions. (Mike Sokolov via shalin)

* SOLR-5861: Recovery should not set onlyIfLeaderActive=true for slice in

'recovery'
state. (shalin)

* SOLR-5423: CSV output doesn't include function field

(Arun Kumar, hossman, Steve Rowe)

* SOLR-5550: shards.info is not returned by a short circuited distributed query.

(Timothy Potter, shalin)

* SOLR-5777: Fix ordering of field values in JSON updates where

field name key is repeated (hossman)

* SOLR-5734: We should use System.nanoTime rather than System.currentTimeMillis

when calculating elapsed time. (Mark Miller, Ramkumar Aiyengar)

* SOLR-5760: ConcurrentUpdateSolrServer has a blockUntilFinished call when

streamDeletes is true that should be tucked into the if statement below it.
 (Mark Miller, Gregory Chanan)

* SOLR-5761: HttpSolrServer has a few fields that can be set via setters but
are not volatile. (Mark Miller, Gregory Chanan)

* SOLR-5907: The hdfs write cache can cause a reader to see a corrupted state.
It now defaults to off, and if you were using solr.hdfs.blockcache.write.enabled
explicitly, you should set it to false.
(Mark Miller)

* SOLR-5811: The Overseer will retry work items until success, which is a serious
problem if you hit a bad work item. (Mark Miller)

* SOLR-5796: Increase how long we are willing to wait for a core to see the ZK
advertised leader in its local state. (Timothy Potter, Mark Miller)

* SOLR-5834: Overseer threads are only being interrupted and not closed.
(hossman, Mark Miller)

* SOLR-5839: ZookeeperInfoServlet does not trim path properly.

(Furkan KAMACI via Mark Miller)

* SOLR-5874: Unsafe cast in CloudSolrServer's RouteException. Change

RouteException to handle Throwable rather than Exception.
(Mark Miller, David Arthur)

* SOLR-5899: CloudSolrServer's RouteResponse and RouteException should be

publicly accessible. (Mark Miller, shalin)

* SOLR-5905: CollapsingQParserPlugin throws a NPE if required 'field' param is

missing.
(Spyros Kapnissis via shalin)

* SOLR-5906: Collection create API ignores property.instanceDir parameter.

(Varun Thacker, shalin)

* SOLR-5920: Distributed sort on DateField, BoolField and BCD{Int,Long,Str}Field

returns string cast exception (Eric Bus, AJ Lemke, hossman, Steve Rowe)

Other Changes
---------------------

* SOLR-5796: Make how long we are willing to wait for a core to see the ZK
advertised leader in its local state configurable.
(Timothy Potter via Mark Miller)

================== 4.7.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.8.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Upgrading from Solr 4.6.0

----------------------
* CloudSolrServer and LBHttpSolrServer no longer declare MalformedURLException
as thrown from their constructors.

* Due to a bug in previous versions the default value of the 'discountOverlap'

property
of DefaultSimilarity was not being set appropriately if you were using the
implicit
DefaultSimilarityFactory instead of explicitly configuring it. To preserve
consistent behavior for people who upgrade, the implicit behavior is now
contingent
on the <luceneMatchVersion/> -- discountOverlap=false for 4.6 and below,
discountOverlap=true for 4.7 and above. See SOLR-5561 for more information.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-5308: SOLR-5601: SOLR-5710: A new 'migrate' collection API to split all

documents with a route key into another collection (shalin)

* SOLR-5441: Expose number of transaction log files and their size via JMX.
(Rafał Kuć via shalin)

* SOLR-5320: Added support for tri-level compositeId routing.

(Anshum Gupta via shalin)

* SOLR-5458: Admin UI - Added a new "Files" conf directory browser/file viewer.

(steffkes)

* SOLR-5447, SOLR-5490: Add a QParserPlugin for Lucene's SimpleQueryParser.

(Jack Conradson via shalin)

* SOLR-5208: Support for the setting of core.properties key/values at create-time

on
Collections API (Erick Erickson)

* SOLR-5428: SOLR-5690: New 'stats.calcdistinct' parameter in StatsComponent

returns
set of distinct values and their count. This can also be specified per field
e.g. 'f.field.stats.calcdistinct'. (Elran Dvir via shalin)

* SOLR-5378, SOLR-5528: A new SuggestComponent that fully utilizes the Lucene

suggester
module and adds pluggable dictionaries, payloads and better distributed support.
This is intended to eventually replace the Suggester support through the
SpellCheckComponent. (Areek Zillur, Varun Thacker via shalin)

* SOLR-5492: Return the replica that actually served the query in shards.info
response. (shalin)

* SOLR-5506: Support docValues in CollationField and ICUCollationField.

(Robert Muir)

* SOLR-5023: Add support for deleteInstanceDir to be passed from SolrJ for Core
Unload action. (Lyubov Romanchuk, shalin)

* SOLR-1871: The 'map' function query accepts a ValueSource as target and

 default value. (Chris Harris, shalin)

* SOLR-5556: Allow class of CollectionsHandler and InfoHandler to be specified

in solr.xml. (Gregory Chanan, Alan Woodward)

* SOLR-5581: Give ZkCLI the ability to get files. (Gregory Chanan via Mark Miller)

* SOLR-5536: Add ValueSource collapse criteria to CollapsingQParsingPlugin (Joel

Bernstein)

* SOLR-5541: Allow QueryElevationComponent to accept elevateIds and excludeIds

as http parameters (Joel Bernstein)

* SOLR-5463: new 'cursorMark' request param for deep paging of sorted result sets
(sarowe, hossman)

* SOLR-5529: Add support for queries to use multiple suggesters.

(Areek Zillur, Erick Erickson, via Robert Muir)

* SOLR-1301: Add a Solr contrib that allows for building Solr indexes via
Hadoop's MapReduce. (Matt Revelle, Alexander Kanarsky, Steve Rowe,
Mark Miller, Greg Bowyer, Jason Rutherglen, Kris Jirapinyo, Jason Venner ,
Andrzej Bialecki, Patrick Hunt, Wolfgang Hoschek, Roman Shaposhnik,
Eric Wong)

* SOLR-5631: Add support for Lucene's FreeTextSuggester.

(Areek Zillur via Robert Muir)

* SOLR-5695: Add support for Lucene's BlendedInfixSuggester.

(Areek Zillur)

* SOLR-5476: Overseer Role for nodes (Noble Paul)

* SOLR-5594: Allow FieldTypes to specify custom PrefixQuery behavior

(Anshum Gupta via hossman)

* LUCENE-5395: Upgrade to Spatial4j 0.4. Various new options are now exposed
automatically for an RPT field type. See Spatial4j CHANGES & javadocs.
https://github.com/spatial4j/spatial4j/blob/master/CHANGES.md (David Smiley)

* SOLR-5670: allow _version_ to use DocValues. (Per Steffensen via yonik)

* SOLR-5535: Set "partialResults" header for shards that error out if

shards.tolerant is specified. (Steve Davids via shalin)

* SOLR-5610: Support cluster-wide properties with an API called CLUSTERPROP (Noble

Paul)

* SOLR-5623: Better diagnosis of RuntimeExceptions in analysis

(Benson Margulies)

* SOLR-5530: Added a NoOpResponseParser for SolrJ which puts the entire raw
response into an entry in the NamedList.
(Upayavira, Vitaliy Zhovtyuk via shalin)

* SOLR-5682: Make the admin InfoHandler more pluggable / derivable.

(Greg Chanan via Mark Miller)

* SOLR-5672: Add logParamsList parameter to support reduced logging.

 (Christine Poerschke via Mark Miller)

* SOLR-3854: SSL support for SolrCloud. (Sami Siren, hossman, Steve Davids,
Alexey Serba, Mark Miller)

Bug Fixes
----------------------

* SOLR-5438: DebugComponent throws NPE when used with grouping.

(Tomás Fernández Löbbe via shalin)

* SOLR-4612: Admin UI - Analysis Screen contains empty table-columns (steffkes)

* SOLR-5451: SyncStrategy closes its http connection manager before the

executor that uses it in its close method. (Mark Miller)

* SOLR-5460: SolrDispatchFilter#sendError can get a SolrCore that it does not

close. (Mark Miller)

* SOLR-5461: Request proxying should only set con.setDoOutput(true) if the

request is a post. (Mark Miller)

* SOLR-5481: SolrCmdDistributor should not let the http client do its own
retries. (Mark Miller)

* LUCENE-5347: Fixed Solr's Zookeeper Client to copy files to Zookeeper using

binary transfer. Previously data was read with default encoding and stored
in zookeeper as UTF-8. This bug was found after upgrading to forbidden-apis
1.4. (Uwe Schindler)

* SOLR-4376: DataImportHandler uses wrong date format for last_index_time if

a delta-import is run first before any full-imports.
(Sebastien Lorber, Arcadius Ahouansou via shalin)

* SOLR-5494: CoreContainer#remove throws NPE rather than returning null when

a SolrCore does not exist in core discovery mode. (Mark Miller)

* SOLR-5354: Distributed sort is broken with CUSTOM FieldType.

(Steve Rowe, hossman, Robert Muir, Jessica Cheng)

* SOLR-5515: NPE when getting stats on date field with empty result on
SolrCloud. (Alexander Sagen, shalin)

* SOLR-5204: StatsComponent and SpellCheckComponent do not support the

shards.tolerant=true parameter. (Anca Kopetz, shalin)

* SOLR-5527: DIH logs spurious warning for special commands. (shalin)

* SOLR-5524: Exception when using Query Function inside Scale Function.

(Trey Grainger, yonik)

* SOLR-5562: ConcurrentUpdateSolrServer constructor ignores supplied httpclient.

(Kyle Halliday via Mark Miller)

* SOLR-5567: ZkController getHostAddress duplicates url prefix.

(Kyle Halliday, Alexey Serba, shalin)

* SOLR-4992: Solr eats OutOfMemoryError exceptions in many cases.

(Mark Miller, Daniel Collins)
* LUCENE-5399, SOLR-5354 sort wouldn't work correctly with
distributed searching for some field types such as legacy numeric
types (Rob Muir, Mike McCandless)

* SOLR-5643: ConcurrentUpdateSolrServer will sometimes not spawn a new Runner

thread even though there are updates in the queue. (Mark Miller)

* SOLR-5650: When a replica becomes a leader, only peer sync with other replicas
that last published an ACTIVE state. (Mark Miller)

* SOLR-5657: When a SolrCore starts on HDFS, it should gracefully handle HDFS

being in safe mode. (Mark Miller)

* SOLR-5663: example-DIH uses non-existing column for mapping (case-sensitive)

(steffkes)

* SOLR-5666: Using the hdfs write cache can result in appearance of corrupted
index. (Mark Miller)

* SOLR-5230: Call DelegatingCollector.finish() during grouping.

(Joel Bernstein, ehatcher)

* SOLR-5679: Shard splitting fails with ClassCastException on collections

upgraded from 4.5 and earlier versions. (Brett Hoerner, shalin)

* SOLR-5673: HTTPSolrServer doesn't set own property correctly in

setFollowRedirects. (Frank Wesemann via shalin)

* SOLR-5676: SolrCloud updates rejected if talking to secure ZooKeeper.

(Greg Chanan via Mark Miller)

* SOLR-5634: SolrJ GroupCommand.getNGroups returns null if group.format=simple

and group.ngroups=true. (Artem Lukanin via shalin)

* SOLR-5667: Performance problem when not using hdfs block cache. (Mark Miller)

* SOLR-5526: Fixed NPE that could arise when explicitly configuring some built
in QParserPlugins (Nikolay Khitrin, Vitaliy Zhovtyuk, hossman)

* SOLR-5598: LanguageIdentifierUpdateProcessor ignores all but the first value

of multiValued string fields. (Andreas Hubold, Vitaliy Zhovtyuk via shalin)

* SOLR-5593: Replicas should accept the last updates from a leader that has just
lost its connection to ZooKeeper. (Christine Poerschke via Mark Miller)

* SOLR-5678: SolrZkClient should throw a SolrException when connect times out

rather than a RuntimeException. (Karl Wright, Anshum Gupta, Mark Miller)

* SOLR-4072: Error message is incorrect for linkconfig in ZkCLI.

(Vamsee Yarlagadda, Adam Hahn, via Mark Miller)

* SOLR-5691: Sharing non thread safe WeakHashMap across thread can cause
problems. (Bojan Smid, Mark Miller)

* SOLR-5693: Running on HDFS does work correctly with NRT search. (Mark Miller)

* SOLR-5644: SplitShard does not handle not finding a shard leader well.
(Mark Miller, Anshum Gupta via shalin)
* SOLR-5704: coreRootDirectory was not respected when creating new cores
via CoreAdminHandler (Jesse Sipprell, Alan Woodward)

* SOLR-5709: Highlighting grouped duplicate docs from different shards with

group.limit > 1 throws ArrayIndexOutOfBoundsException. (Steve Rowe)

* SOLR-5561: Fix implicit DefaultSimilarityFactory initialization in IndexSchema

to properly specify discountOverlap option.
(Isaac Hebsh, Ahmet Arslan, Vitaliy Zhovtyuk, hossman)

* SOLR-5689: On reconnect, ZkController cancels election on first context rather

than latest. (Gregory Chanan, Mark Miller via shalin)

* SOLR-5649: Clean up some minor ConnectionManager issues.

(Mark Miller, Gregory Chanan)

* SOLR-5365: Fix bug with compressed files in ExtractingRequestHandler by

upgrading commons-compress to 1.7 (Jan Høydahl, hossman)

* SOLR-5675: cloud-scripts/zkcli.bat: quote option log4j

(Günther Ruck via steffkes

* SOLR-5721: ConnectionManager can become stuck in likeExpired.

(Gregory Chanan via Mark Miller)

* SOLR-5731: In ConnectionManager, we should catch and only log exceptions

from BeforeReconnect. (Mark Miller)

* SOLR-5718: Make LBHttpSolrServer zombie checks non-distrib and non-scoring.

(Christine Poerschke via Mark Miller)

* SOLR-5727: LBHttpSolrServer should only retry on Connection exceptions when

sending updates. Affects CloudSolrServer. (Mark Miller)

* SOLR-5739: Sub-shards created by shard splitting have their update log set
to buffering mode on restarts. (Günther Ruck, shalin)

* SOLR-5741: UpdateShardHandler was not correctly setting max total connections

on the HttpClient. (Shawn Heisey)

* SOLR-5620: ZKStateReader.aliases should be volatile to ensure all threads see

the latest aliases. (Ramkumar Aiyengar via Mark Miller)

* SOLR-5448: ShowFileRequestHandler treats everything as Directory, when in

Cloud-Mode. (Erick Erickson, steffkes)

Optimizations
----------------------

* SOLR-5436: Eliminate the 1500ms wait in overseer loop as well as

polling the ZK distributed queue. (Noble Paul, Mark Miller)

* SOLR-5189: Solr 4.x Web UI Log Viewer does not display 'date' column from
logs (steffkes)

* SOLR-5512: Optimize DocValuesFacets. (Robert Muir)

* SOLR-2960: fix DIH XPathEntityProcessor to add the correct number of "null"

 placeholders for multi-valued fields (Michael Watts via James Dyer)

* SOLR-5214: Reduce memory usage for shard splitting by merging segments one
at a time. (Christine Poerschke via shalin)

* SOLR-4227: Wrap XML RequestWriter's OutputStreamWriter in a BufferedWriter

to avoid frequent converter invocations. (Conrad Herrmann, shalin)

* SOLR-5624: Enable QueryResultCache for CollapsingQParserPlugin.

(David Boychuck, Joel Bernstein)

* LUCENE-5440: DocSet decoupled from OpenBitSet. DocSetBase moved to use

FixedBitSet instead of OpenBitSet. As a result BitDocSet now only works
with FixedBitSet. (Shai Erera)

Other Changes
---------------------

* SOLR-5399: Add distributed request tracking information to DebugComponent

(Tomás Fernández Löbbe via Ryan Ernst)

* SOLR-5421: Remove double set of distrib.from param in processAdd method of

DistributedUpdateProcessor. (Anshum Gupta via shalin)

* SOLR-5404: The example config references deprecated classes.

(Uwe Schindler, Rafał Kuć via Mark Miller)

* SOLR-5487: Replication factor error message doesn't match constraint.

(Patrick Hunt via shalin)

* SOLR-5499: Log a warning if /get is not registered when using SolrCloud.

(Daniel Collins via shalin)

* SOLR-5517: Return HTTP error on POST requests with no Content-Type.

(Ryan Ernst, Uwe Schindler)

* SOLR-5502: Added a test for tri-level compositeId routing with documents

having a "/" in a document id. (Anshum Gupta via Mark Miller)

* SOLR-5533: Improve out of the box support for running Solr on hdfs with
SolrCloud. (Mark Miller)

* SOLR-5548: Give DistributedSearchTestCase / JettySolrRunner the ability to

specify extra filters. (Greg Chanan via Mark Miller)

* SOLR-5555: LBHttpSolrServer and CloudSolrServer constructors don't need to

declare MalformedURLExceptions (Sushil Bajracharya, Alan Woodward)

* SOLR-5565: Raise default ZooKeeper session timeout to 30 seconds from 15

seconds. (Mark Miller)

* SOLR-5574: CoreContainer shutdown publishes all nodes as down and waits to

see that and then again publishes all nodes as down. (Mark Miller)

* SOLR-5590: Upgrade HttpClient/HttpComponents to 4.3.x.

(Karl Wright via Shawn Heisey)

* SOLR-2794: change the default of hl.phraseLimit to 5000.

(Michael Della Bitta via Robert Muir, Koji, zarni)
* SOLR-5632: Improve response message for reloading a non-existent core.
(Anshum Gupta via Mark Miller)

* SOLR-5633: HttpShardHandlerFactory should make its http client available to

subclasses.
(Ryan Ernst)

* SOLR-5684: Shutdown SolrServer clients created in BasicDistributedZk2Test and

BasicDistributedZkTest. (Tomás Fernández Löbbe via shalin)

* SOLR-5629: SolrIndexSearcher.name should include core name.

(Shikhar Bhushan via shalin)

* SOLR-5702: Log config name found for collection at info level.

(Christine Poerschke via Mark Miller)

* SOLR-5659: Add test for compositeId ending with an '!'.

(Markus Jelsma, Anshum Gupta via shalin)

* SOLR-5700: Improve error handling of remote queries (proxied requests).

(Greg Chanan, Steve Davids via Mark Miller)

* SOLR-5585: Raise Collections API timeout to 3 minutes from one minute.

(Mark Miller)

* SOLR-5257: Improved error/warn messages when Update XML contains unexpected XML
nodes
(Vitaliy Zhovtyuk, hossman)

================== 4.6.1 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.8.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Detailed Change List

----------------------

Bug Fixes
----------------------

* SOLR-5408: CollapsingQParserPlugin scores incorrectly when multiple sort criteria

are used
(Brandon Chapman, Joel Bernstein)

* SOLR-5416: CollapsingQParserPlugin breaks Tag/Exclude Faceting (David Boychuck,

Joel Bernstein)

* SOLR-5442: Python client cannot parse proxied response when served by Tomcat.
(Patrick Hunt, Gregory Chanan, Vamsee Yarlagadda, Romain Rigaux, Mark Miller)

* SOLR-5445: Proxied responses should propagate all headers rather than the
first one for each key. (Patrick Hunt, Mark Miller)
* SOLR-5479: SolrCmdDistributor retry logic stops if a leader for the request
cannot be found in 1 second. (Mark Miller)

* SOLR-5532: SolrJ Content-Type validation is too strict for some

webcontainers / proxies. (Jakob Furrer, hossman, Shawn Heisey, Uwe Schindler,
Mark Miller)

* SOLR-5547: Creating a collection alias using SolrJ's CollectionAdminRequest

sets the alias name and the collections to alias to the same value.
(Aaron Schram, Mark Miller)

* SOLR-5577: Likely ZooKeeper expiration should not slow down updates a given
amount, but instead cut off updates after a given time.
(Mark Miller, Christine Poerschke, Ramkumar Aiyengar)

* SOLR-5580: NPE when creating a core with both explicit shard and coreNodeName.
(YouPeng Yang, Mark Miller)

* SOLR-5552: Leader recovery process can select the wrong leader if all replicas
for a shard are down and trying to recover as well as lose updates that should
have been recovered. (Timothy Potter, Mark Miller)

* SOLR-5569 A replica should not try and recover from a leader until it has
published that it is ACTIVE. (Mark Miller)

* SOLR-5568 A SolrCore cannot decide to be the leader just because the cluster
state says no other SolrCore's are active. (Mark Miller)

* SOLR-5496: We should share an http connection manager across non search

HttpClients and ensure all http connection managers get shutdown.
(Mark Miller)

* SOLR-5583: ConcurrentUpdateSolrServer#blockUntilFinished may wait forever if

the executor service is shutdown. (Mark Miller)

* SOLR-5586: All ZkCmdExecutor's should be initialized with the zk client

timeout. (Mark Miller)

* SOLR-5587: ElectionContext implementations should use

ZkCmdExecutor#ensureExists to ensure their election paths are properly
created. (Mark Miller)

* SOLR-5540: HdfsLockFactory should explicitly create the lock parent directory if

necessary. (Mark Miller)

* SOLR-4709: The core reload after replication if config files have changed
can fail due to a race condition. (Mark Miller, Hossman)

* SOLR-5503: Retry 'forward to leader' requests less aggressively - rather

than on IOException and status 500, ConnectException. (Mark Miller)

* SOLR-5588: PeerSync doesn't count all connect failures as success.

(Mark Miller)

* SOLR-5564: hl.maxAlternateFieldLength should apply to original field when

fallback is attempted (janhoy)

* SOLR-5608: Don't allow a closed SolrCore to publish state to ZooKeeper.

 (Mark Miller, Shawn Heisey)

* SOLR-5615: Deadlock while trying to recover after a ZK session expiration.

(Ramkumar Aiyengar, Mark Miller)

* SOLR-5543: Core swaps resulted in duplicate core entries in solr.xml when

using solr.xml persistence. (Bill Bell, Alan Woodward)

* SOLR-5618: Fix false cache hits in queryResultCache when hashCodes are equal
and duplicate filter queries exist in one of the requests (hossman)

* SOLR-4260: ConcurrentUpdateSolrServer#blockUntilFinished can return before

all previously added updates have finished. This could cause distributed
updates meant for replicas to be lost. (Markus Jelsma, Timothy Potter,
Joel Bernstein, Mark Miller)

* SOLR-5645: A SolrCore reload via the CoreContainer will try and register in
zk again with the new SolrCore. (Mark Miller)

* SOLR-5636: SolrRequestParsers does some xpath lookups on every request, which

can cause concurrency issues. (Mark Miller)

* SOLR-5658: commitWithin and overwrite are not being distributed to replicas

now that SolrCloud uses javabin to distribute updates.
(Mark Miller, Varun Thacker, Elodie Sannier, shalin)

Optimizations
----------------------

* SOLR-5576: Improve concurrency when registering and waiting for all

SolrCore's to register a DOWN state. (Christine Poerschke via Mark Miller)

================== 4.6.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.8.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Upgrading from Solr 4.5.0

----------------------

* If you are using methods from FieldMutatingUpdateProcessorFactory for getting

configuration information (oneOrMany or getBooleanArg), those methods have
been moved to NamedList and renamed to removeConfigArgs and removeBooleanArg,
respectively. The original methods are deprecated, to be removed in 5.0.
See SOLR-5264.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-5167: Add support for AnalyzingInfixSuggester (AnalyzingInfixLookupFactory).

(Areek Zillur, Varun Thacker via Robert Muir)
* SOLR-5246: Shard splitting now supports collections configured with router.field.
(shalin)

* SOLR-5274: Allow JettySolrRunner SSL config to be specified via a constructor.

(Mark Miller)

* SOLR-5300: Shards can be split by specifying arbitrary number of hash ranges

within the shard's hash range. (shalin)

* SOLR-5226: Add Lucene index heap usage to the Solr admin UI.
(Areek Zillur via Robert Muir)

* SOLR-5324: Make sub shard replica recovery and shard state switch asynchronous.
(Yago Riveiro, shalin)

* SOLR-5338: Split shards by a route key using split.key parameter. (shalin)

* SOLR-5353: Enhance CoreAdmin api to split a route key's documents from an index
and leave behind all other documents. (shalin)

* SOLR-5027: CollapsingQParserPlugin for high performance field collapsing on high

cardinality fields.
(Joel Bernstein)

* SOLR-5395: Added a RunAlways marker interface for UpdateRequestProcessorFactory

implementations indicating that they should not be removed in later stages
of distributed updates (usually signalled by the update.distrib parameter)
(yonik)

* SOLR-5310: Add a collection admin command to remove a replica (noble)

* SOLR-5311: Avoid registering replicas which are removed (noble)

* SOLR-5406: CloudSolrServer failed to propagate request parameters

along with delete updates. (yonik)

* SOLR-5374: Support user configured doc-centric versioning rules

via the optional DocBasedVersionConstraintsProcessorFactory
update processor (Hossman, yonik)

* SOLR-5392: Extend solrj apis to cover collection management.

(Roman Shaposhnik via Mark Miller)

* SOLR-5084: new field type EnumField. (Elran Dvir via Erick Erickson)

* SOLR-5464: Add option to ConcurrentSolrServer to stream pure delete

requests. (Mark Miller)

Bug Fixes
----------------------

* SOLR-5216: Document updates to SolrCloud can cause a distributed deadlock.

(Mark Miller)

* SOLR-5367: Unmarshalling delete by id commands with JavaBin can lead to class

cast
exception. (Mark Miller)
* SOLR-5359: ZooKeeper client is not closed when it fails to connect to an
ensemble.
(Mark Miller, Klaus Herrmann)

* SOLR-5042: MoreLikeThisComponent was using the rows/count value in place of

flags, which caused a number of very strange issues, including NPEs and
ignoring requests for the results to include the score.
(Anshum Gupta, Mark Miller, Shawn Heisey)

* SOLR-5371: Solr should consistently call SolrServer#shutdown (Mark Miller)

* SOLR-5363: Solr doesn't start up properly with Log4J2 (Petar Tahchiev via Alan
Woodward)

* SOLR-5380: Using cloudSolrServer.setDefaultCollection(collectionId) does not

work as intended for an alias spanning more than 1 collection.
(Thomas Egense, Shawn Heisey, Mark Miller)

* SOLR-5418: Background merge after field removed from solr.xml causes error.
(Reported on user's list, Robert M's patch via Erick Erickson)

* SOLR-5318: Creating a core via the admin API doesn't respect transient property
(Olivier Soyez via Erick Erickson)

* SOLR-5388: Creating a new core via the HTTP API that results in a transient being
unloaded results in a " Too many close [count:-1]" error.
(Olivier Soyez via Erick Erickson)

* SOLR-5453: Raise recovery socket read timeouts. (Mark Miller)

* SOLR-5397: Replication can fail silently in some cases. (Mark Miller)

* SOLR-5465: SolrCmdDistributor retry logic has a concurrency race bug.

(Mark Miller)

* SOLR-5452: Do not attempt to proxy internal update requests. (Mark Miller)

Optimizations
----------------------

* SOLR-5232: SolrCloud should distribute updates via streaming rather than

buffering.
(Mark Miller)

* SOLR-5223: SolrCloud should use the JavaBin binary format for communication by
default.
(Mark Miller)

* SOLR-5370: Requests to recover when an update fails should be done in

background threads. (Mark Miller)

* LUCENE-5300,LUCENE-5304: Specialized faceting for fields which are declared as

multi-valued in the schema but are actually single-valued. (Adrien Grand)

Security
----------------------

* SOLR-4882: SolrResourceLoader was restricted to only allow access to resource

files below the instance dir. The reason for this is security related: Some
 Solr components allow to pass in resource paths via REST parameters
(e.g. XSL stylesheets, velocity templates,...) and load them via resource
loader. For backwards compatibility, this security feature can be disabled
by a new system property: solr.allow.unsafe.resourceloading=true
(Uwe Schindler)

Other Changes
----------------------

* SOLR-5237: Add indexHeapUsageBytes to LukeRequestHandler, indicating how much

heap memory is being used by the underlying Lucene index structures.
(Areek Zillur via Robert Muir)

* SOLR-5241: Fix SimplePostToolTest performance problem - implicit DNS lookups

(hossman)

* SOLR-5273: Update HttpComponents to 4.2.5 and 4.2.6. (Mark Miller)

* SOLR-5264: Move methods for getting config information from

FieldMutatingUpdateProcessorFactory to NamedList. (Shawn Heisey)

* SOLR-5319: Remove unused and incorrect router name from Collection ZK nodes.
(Jessica Cheng via shalin)

* SOLR-5321: Remove unnecessary code in Overseer.updateState method which tries to

use router name from message where none is ever sent. (shalin)

* SOLR-5401: SolrResourceLoader logs a warning if a deprecated (factory) class

is used in schema or config. (Uwe Schindler)

* SOLR-3397: Warn if master or slave replication is enabled in SolrCloud mode.

(Erick
Erickson)

================== 4.5.1 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.8.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Detailed Change List

----------------------

Bug Fixes
----------------------

* SOLR-4590: Collections API should return a nice error when not in SolrCloud mode.
(Anshum Gupta, Mark Miller)

* SOLR-5295: The CREATESHARD collection API creates maxShardsPerNode number of

replicas if replicationFactor is not specified. (Brett Hoerner, shalin)

* SOLR-5296: Creating a collection with implicit router adds shard ranges

to each shard. (shalin)
* SOLR-5263: Fix CloudSolrServer URL cache update race. (Jessica Cheng, Mark
Miller)

* SOLR-5297: Admin UI - Threads Screen missing Icon (steffkes)

* SOLR-5301: DELETEALIAS command prints CREATEALIAS in logs (janhoy)

* SOLR-5255: Remove unnecessary call to fetch and watch live nodes in ZkStateReader
cluster watcher. (Jessica Cheng via shalin)

* SOLR-5305: Admin UI - Reloading System-Information on Dashboard does not work

anymore (steffkes)

* SOLR-5314: Shard split action should use soft commits instead of hard commits
to make sub shard data visible. (Kalle Aaltonen, shalin)

* SOLR-5327: SOLR-4915, "The root cause should be returned to the user when a
SolrCore create
call fails", was reverted. (Mark Miller)

* SOLR-5317: SolrCore persistence bugs if defining SolrCores in solr.xml.

(Mark Miller, Yago Riveiro)

* SOLR-5306: Extra collection creation parameters like collection.configName are

not being respected. (Mark Miller, Liang Tianyu, Nathan Neulinger)

* SOLR-5325: ZooKeeper connection loss can cause the Overseer to stop processing
commands. (Christine Poerschke, Mark Miller, Jessica Cheng)

* SOLR-4327: HttpSolrServer can leak connections on errors. (Karl Wright, Mark

Miller)

* SOLR-5349: CloudSolrServer - ZK timeout arguments passed to ZkStateReader are

flipped.
(Ricardo Merizalde via shalin)

* SOLR-5330: facet.method=fcs on single values fields could sometimes result

in incorrect facet labels. (Michael Froh, yonik)

Other Changes
----------------------

* SOLR-5323: Disable ClusteringComponent by default in collection1 example.

The solr.clustering.enabled system property needs to be set to 'true'
to enable the clustering contrib (reverts SOLR-4708). (Dawid Weiss)

================== 4.5.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.8.0
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Upgrading from Solr 4.4.0

----------------------
* XML configuration parsing is now more strict about situations where a single
setting is allowed but multiple values are found. In the past, one value
would be chosen arbitrarily and silently. Starting with 4.5, configuration
parsing will fail with an error in situations like this. If you see error
messages such as "solrconfig.xml contains more than one value for config path:
XXXXX" or "Found Z configuration sections when at most 1 is allowed matching
expression: XXXXX" check your solrconfig.xml file for multiple occurrences of
XXXXX and delete the ones that you do not wish to use. See SOLR-4953 &
SOLR-5108 for more details.

* In the past, schema.xml parsing would silently ignore "default" or "required"

options specified on <dynamicField/> declarations. Beginning with 4.5,
attempting
to do configured these on a dynamic field will cause an init error. If you
encounter one of these errors when upgrading an existing schema.xml, you can
safely remove these attributes, regardless of their value, from your config and
Solr will continue to behave exactly as it did in previous versions. See
SOLR-5227 for more details.

* The UniqFieldsUpdateProcessorFactory has been improved to support all of the

FieldMutatingUpdateProcessorFactory selector options. The <lst named="fields">
init param option is now deprecated and should be replaced with the more standard
<arr name="fieldName">. See SOLR-4249 for more details.

* UpdateRequestExt has been removed as part of SOLR-4816. You should use

UpdateRequest
instead.

* CloudSolrServer can now use multiple threads to add documents by default. This is
a
small change in runtime semantics when using the bulk add method - you will still
end up with the same exception on a failure, but some documents beyond the one
that
failed may have made it in. To get the old, single threaded behavior, set
parallel updates
to false on the CloudSolrServer instance.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-5219: Rewritten selection of the default search and document clustering

algorithms. (Dawid Weiss)

* SOLR-5202: Support easier overrides of Carrot2 clustering attributes via

XML data sets exported from the Workbench. (Dawid Weiss)

* SOLR-5126: Update Carrot2 clustering to version 3.8.0, update Morfologik

to version 1.7.1 (Dawid Weiss)

* SOLR-2345: Enhanced geodist() to work with an RPT field, provided that the
field is referenced via 'sfield' and the query point is constant.
(David Smiley)

* SOLR-5082: The encoding of URL-encoded query parameters can be changed with

the "ie" (input encoding) parameter, e.g. "select?q=m%FCller&ie=ISO-8859-1".
 The default is UTF-8. To change the encoding of POSTed content, use the
"Content-Type" HTTP header. (Uwe Schindler, Shawn Heisey)

* SOLR-4221: Custom sharding (Noble Paul)

* SOLR-4808: Persist and use router,replicationFactor and maxShardsPerNode at

Collection
and Shard level (Noble Paul, Shalin Mangar)

* SOLR-5006: CREATESHARD command for 'implicit' shards (Noble Paul)

* SOLR-5017: Allow sharding based on the value of a field (Noble Paul)

* SOLR-4222: create custom sharded collection via collections API (Noble Paul)

* SOLR-4718: Allow solr.xml to be stored in ZooKeeper. (Mark Miller, Erick

Erickson)

* SOLR-5156: Enhance ZkCLI to allow uploading of arbitrary files to ZK. (Erick

Erickson)

* SOLR-5165: Single-valued docValues fields no longer require a default value.

Additionally they work with sortMissingFirst, sortMissingLast, facet.missing,
exists() in function queries, etc. (Robert Muir)

* SOLR-5182: Add NoOpRegenerator, a regenerator for custom per-segment caches

where items are preserved across commits. (Robert Muir)

* SOLR-4249: UniqFieldsUpdateProcessorFactory now extends

FieldMutatingUpdateProcessorFactory and supports all of its selector options. Use
of the "fields" init param is now deprecated in favor of "fieldName" (hossman)

* SOLR-2548: Allow multiple threads to be specified for faceting. When threading,

one
can specify facet.threads to parallelize loading the uninverted fields. In at
least
one extreme case this reduced warmup time from 20 seconds to 3 seconds. (Janne
Majaranta,
Gun Akkor via Erick Erickson, David Smiley)

* SOLR-4816: CloudSolrServer can now route updates locally and no longer relies on
inter-node
update forwarding. (Joel Bernstein, Shikhar Bhushan, Stephen Riesenberg, Mark
Miller)

* SOLR-3249: Allow CloudSolrServer and SolrCmdDistributor to use JavaBin. (Mark

Miller)

Bug Fixes
----------------------

* SOLR-3633: web UI reports an error if CoreAdminHandler says there are no

SolrCores (steffkes)

* SOLR-4489: SpellCheckComponent can throw StringIndexOutOfBoundsException

when generating collations involving multiple word-break corrections.
(James Dyer)

* SOLR-5087 - CoreAdminHandler.handleMergeAction generating NullPointerException

 (Patrick Hunt via Erick Erickson)

* SOLR-5107: Fixed NPE when using numTerms=0 in LukeRequestHandler

(Ahmet Arslan, hossman)

* SOLR-4679, SOLR-4908, SOLR-5124: Text extracted from HTML or PDF files

using Solr Cell was missing ignorable whitespace, which is inserted by
TIKA for convenience to support plain text extraction without using the
HTML elements. This bug resulted in glued words. (hossman, Uwe Schindler)

* SOLR-5121: zkcli usage help for makepath doesn't match actual command.
(Daniel Collins via Mark Miller)

* SOLR-5119: Managed schema problems after adding fields via Schema Rest API.
(Nils Kübler, Steve Rowe)

* SOLR-5133: HdfsUpdateLog can fail to close a FileSystem instance if init

is called more than once. (Mark Miller)

* SOLR-5135: Harden Collection API deletion of /collections/$collection

ZooKeeper node. (Mark Miller)

* SOLR-4764: When using NRT, just init the first reader from IndexWriter.
(Robert Muir, Mark Miller)

* SOLR-5122: Fixed bug in spellcheck.collateMaxCollectDocs. Eliminates risk

of divide by zero, and makes estimated hit counts meaningful in non-optimized
indexes. (hossman)

* SOLR-3936: Fixed QueryElevationComponent sorting when used with Grouping

(Michael Garski via hossman)

* SOLR-5171: SOLR Admin gui works in IE9, breaks in IE10. (Joseph L Howard via
steffkes)

* SOLR-5174: Admin UI - Query View doesn't highlight (json) Result if it

contains HTML Tags (steffkes)

* SOLR-4817 Solr should not fall back to the back compat built in solr.xml in
SolrCloud
mode (Erick Erickson)

* SOLR-5112: Show full message in Admin UI Logging View (Matthew Keeney via
steffkes)

* SOLR-5190: SolrEntityProcessor substitutes variables only once in child entities

(Harsh Chawla, shalin)

* SOLR-3852: Fixed ZookeeperInfoServlet so that the SolrCloud Admin UI pages will

work even if ZK contains nodes with data which are not utf8 text. (hossman)

* SOLR-5206: Fixed OpenExchangeRatesOrgProvider to use refreshInterval correctly

(Catalin, hossman)

* SOLR-5215: Fix possibility of deadlock in ZooKeeper ConnectionManager.

(Mark Miller, Ricardo Merizalde)

* SOLR-4909: Use DirectoryReader.openIfChanged in non-NRT mode.

(Michael Garski via Robert Muir)
* SOLR-5227: Correctly fail schema initialization if a dynamicField is configured
to
be required, or have a default value. (hossman)

* SOLR-5231: Fixed a bug with the behavior of BoolField that caused documents w/o
a value for the field to act as if the value were true in functions if no other
documents in the same index segment had a value of true.
(Robert Muir, hossman, yonik)

* SOLR-5233: The "deleteshard" collections API doesn't wait for cluster state to
update,
can fail if some nodes of the deleted shard were down and had incorrect logging.
(Christine Poerschke, shalin)

* SOLR-5150: HdfsIndexInput may not fully read requested bytes. (Mark Miller,
Patrick Hunt)

* SOLR-5240: All solr cores will now be loaded in parallel (as opposed to a fixed
number)
in zookeeper mode to avoid deadlocks due to replicas waiting for other replicas
to come up. (yonik)

* SOLR-5243: Killing a shard in one collection can result in leader election in a

different
collection if they share the same coreNodeName. (yonik, Mark Miller)

* SOLR-5281: IndexSchema log message was printing '[null]' instead of

'[<core name>]' (Jun Ohtani via Steve Rowe)

* SOLR-5279: Implicit properties don't seem to exist on core RELOAD

(elyograg, hossman, Steve Rowe)

* SOLR-5291: Solrj does not propagate the root cause to the user for many errors.
(Mark Miller)

Optimizations
----------------------

* SOLR-5044: Admin UI - Note on Core-Admin about directories while creating

core (steffkes)

* SOLR-5134: Have HdfsIndexOutput extend BufferedIndexOutput.

(Mark Miller, Uwe Schindler)

* SOLR-5057: QueryResultCache should not related with the order of fq's list
(Feihong Huang via Erick Erickson)

* SOLR-4816: CloudSolrServer now uses multiple threads to send updates by default.

(Joel Bernstein via Mark Miller)

* SOLR-3530: Better error messages / Content-Type validation in SolrJ. (Mark

Miller, hossman)

Other Changes
----------------------

* SOLR-4708: Enable ClusteringComponent by default in collection1 example.

The solr.clustering.enabled system property is set to 'true' by default.
 (ehatcher, Dawid Weiss)

* SOLR-4914, SOLR-5162: Factor out core list persistence and discovery into a
new CoresLocator interface. (Alan Woodward, Shawn Heisey)

* SOLR-5056: Improve type safety of ConfigSolr class. (Alan Woodward)

* SOLR-4951: Better randomization of MergePolicy in Solr tests (hossman)

* SOLR-4953, SOLR-5108: Make XML Configuration parsing fail if an xpath matches

multiple nodes when only a single value or plugin instance is expected.
(hossman)

* The routing parameter "shard.keys" is deprecated as part of SOLR-5017 .The new

parameter name is '_route_' .
The old parameter should continue to work for another release (Noble Paul)

* SOLR-5173: Solr-core's Maven configuration includes test-only Hadoop

dependencies as indirect compile-time dependencies.
(Chris Collins, Steve Rowe)

================== 4.4.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.4
Carrot2 3.6.2
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Upgrading from Solr 4.3.0

----------------------

* TieredMergePolicy and the various subtypes of LogMergePolicy no longer have

an explicit "setUseCompoundFile" method. Instead the behavior of new
segments is determined by the IndexWriter configuration, and the MergePolicy
is only consulted to determine if merge segments should use the compound
file format (based on the value of "setNoCFSRatio"). If you have explicitly
configured one of these classes using <mergePolicy> and include an init arg
like this...
<bool name="useCompoundFile">true</bool>
...this will now be treated as if you specified...
<useCompoundFile>true</useCompoundFile>
...directly on the <indexConfig> (overriding any value already set using that
syntax) and a warning will be logged to updated your configuration. Users
with an explicitly declared <mergePolicy> are encouraged to review the
current javadocs for their MergePolicy subclass and review their configured
options carefully. See SOLR-4941, SOLR-4934 and LUCENE-5038 for more
information.

* SOLR-4778: The signature of LogWatcher.registerListener has changed, from

(ListenerConfig, CoreContainer) to (ListenerConfig). Users implementing their
own LogWatcher classes will need to change their code accordingly.

* LUCENE-5063: ByteField and ShortField have been deprecated and will be removed
in 5.0. If you are still using these field types, you should migrate your
fields to TrieIntField.
Detailed Change List
----------------------

New Features
----------------------

* SOLR-3251: Dynamically add fields to schema. (Steve Rowe, Robert Muir, yonik)

* SOLR-4761, SOLR-4976: Add option to plugin a merged segment warmer into

solrconfig.xml.
Info about segments warmed in the background is available via infostream.
(Mark Miller, Ryan Ernst, Mike McCandless, Robert Muir)

* SOLR-3240: Add "spellcheck.collateMaxCollectDocs" option so that when testing

potential Collations against the index, SpellCheckComponent will only collect
n documents, thereby estimating the hit-count. This is a performance
optimization
in cases where exact hit-counts are unnecessary. Also, when
"collateExtendedResults"
is false, this optimization is always made (James Dyer).

* SOLR-4785: New MaxScoreQParserPlugin returning max() instead of sum() of terms

(janhoy)

* SOLR-4234: Add support for binary files in ZooKeeper. (Eric Pugh via Mark Miller)

* SOLR-4048: Add findRecursive method to NamedList. (Shawn Heisey)

* SOLR-4228: SolrJ's SolrPing object has new methods for ping, enable, and
disable. (Shawn Heisey, hossman, Steve Rowe)

* SOLR-4893: Extend FieldMutatingUpdateProcessor.ConfigurableFieldNameSelector

to enable checking whether a field matches any schema field. To select field
names that don't match any fields or dynamic fields in the schema, add
<bool name="fieldNameMatchesSchemaField">false</bool> to an update
processor's configuration in solrconfig.xml. (Steve Rowe, hossman)

* SOLR-4921: Admin UI now supports adding documents to Solr (gsingers, steffkes)

* SOLR-4916: Add support to write and read Solr index files and transaction log
files to and from HDFS. (phunt, Mark Miller, Gregory Chanan)

* SOLR-4892: Add FieldMutatingUpdateProcessorFactory subclasses

Parse{Date,Integer,Long,Float,Double,Boolean}UpdateProcessorFactory. These
factories have a default selector that matches all fields that either don’t
match any schema field, or are in the schema with the corresponding
typeClass. If they see a value that is not a CharSequence, or can't parse
the value, they leave it as is. For multi-valued fields, these processors
will not convert any values unless all are first successfully parsed, or
already are instances of the target class. Ordering the processors, e.g.
[Boolean, Long, Double, Date] will allow e.g. values ["2", "5", "8.6"] to
be left alone by the Boolean and Long processors, but then converted by the
Double processor. (Steve Rowe, hossman)

* SOLR-4972: Add PUT command to ZkCli tool. (Roman Shaposhnik via Mark Miller)

* SOLR-4973: Adding getter method for defaultCollection on CloudSolrServer.

(Furkan KAMACI via Mark Miller)
* SOLR-4897: Add solr/example/example-schemaless/, an example config set
for schemaless mode. (Steve Rowe)

* SOLR-4655: Add option to have Overseer assign generic node names so that
new addresses can host shards without naming confusion. (Mark Miller, Anshum
Gupta)

* SOLR-4977: Add option to send IndexWriter's infostream to the logging system.

(Ryan Ernst via Robert Muir)

* SOLR-4693: A "deleteshard" collections API that unloads all replicas of a given

shard and then removes it from the cluster state. It will remove only those
shards
which are INACTIVE or have no range (created for custom sharding).
(Anshum Gupta, shalin)

* SOLR-5003: CSV Update Handler supports optionally adding the line number/row id
to
a document (gsingers)

* SOLR-5010: Add support for creating copy fields to the Schema REST API (gsingers)

* SOLR-4991: Register QParserPlugins as SolrInfoMBeans (ehatcher)

* SOLR-4943: Add a new system wide info admin handler that exposes the system info
that could previously only be retrieved using a SolrCore. (Mark Miller)

* SOLR-3076: Block joins. Documents and their sub-documents must be indexed

as a block.
{!parent which=<allParents>}<someChildren> takes in a query that matches child
documents and results in matches on their parents.
{!child of=<allParents>}<someParents> takes in a query that matches some parent
documents and results in matches on their children.
(Mikhail Khludnev, Vadim Kirilchuk, Alan Woodward, Tom Burton-West, Mike
McCandless,
hossman, yonik)

Bug Fixes
----------------------

* SOLR-4333: edismax parser to not double-escape colons if already escaped by

the client application (James Dyer, Robert J. van der Boon)

* SOLR-4776: Solrj doesn't return "between" count in range facets

(Philip K. Warren via shalin)

* SOLR-4616: HitRatio on caches is now exposed over JMX MBeans as a float.

(Greg Bowyer)

* SOLR-4803: Fixed core discovery mode (ie: new style solr.xml) to treat
'collection1' as the default core name. (hossman)

* SOLR-4790: Throw an error if a core has the same name as another core, both old
and
new style solr.xml

* SOLR-4842: Fix facet.field local params from affecting other facet.field's.

 (ehatcher, hossman)

* SOLR-4814: If a SolrCore cannot be created it should remove any information it

published about itself from ZooKeeper. (Mark Miller)

* SOLR-4863: Removed non-existent attribute sourceId from dynamic JMX stats

to fix AttributeNotFoundException (suganuma, hossman via shalin)

* SOLR-4891: JsonLoader should preserve field value types from the JSON content
stream.
(Steve Rowe)

* SOLR-4805: SolrCore#reload should not call preRegister and publish a DOWN state
to
ZooKeeper. (Mark Miller, Jared Rodriguez)

* SOLR-4899: When reconnecting after ZooKeeper expiration, we need to be willing to

wait
forever, not just for 30 seconds. (Mark Miller)

* SOLR-4920: JdbcDataSource incorrectly suppresses exceptions when retrieving a

connection from
a JNDI context and falls back to trying to use DriverManager to obtain a
connection. Additionally,
if a SQLException is thrown while initializing a connection, such as in
setAutoCommit(), the
connection will not be closed. (Chris Eldredge via shalin)

* SOLR-4915: The root cause should be returned to the user when a SolrCore create
call fails.
(Mark Miller)

* SOLR-4925 : Collection create throws NPE when 'numShards' param is missing (Noble
Paul)

* SOLR-4910: persisting solr.xml is broken. More stringent testing of persistence

fixed
up a number of issues and several bugs with persistence. Among them are
- don't persisting implicit properties
- should persist zkHost in the <solr> tag (user's list)
- reloading a core that has transient="true" returned an error. reload should
load
a transient core if it's not yet loaded.
- No longer persisting loadOnStartup or transient core properties if they were
not
specified in the original solr.xml
- Testing flushed out the fact that you couldn't swap a core marked
transient=true
loadOnStartup=false because it hadn't been loaded yet.
- SOLR-4862, CREATE fails to persist schema, config, and dataDir
- SOLR-4363, not persisting coreLoadThreads in <solr> tag
- SOLR-3900, logWatcher properties not persisted
- SOLR-4850, cores defined as loadOnStartup=true, transient=false can't be
searched
(Erick Erickson)

* SOLR-4923: Commits to non leaders as part of a request that also contain updates
can execute out of order. (hossman, Ricardo Merizalde, Mark Miller)
* SOLR-4932: persisting solr.xml saves some parameters it shouldn't when they
weren't
defined in the original. Benign since the default values are saved, but still
incorrect.
(Erick Erickson, thanks Shawn Heisey for helping test!)

* SOLR-4934, SOLR-4941: Fix handling of <mergePolicy> init arg

"useCompoundFile" needed after changes in LUCENE-5038 (hossman)

* SOLR-4456: Admin UI: Displays dashboard even if Solr is down (steffkes)

* SOLR-4949: UI Analysis page dropping characters from input box (steffkes)

* SOLR-4960: Fix race conditions in shutdown of CoreContainer

and getCore that could cause a request to attempt to use a core that
has shut down. (yonik)

* SOLR-4926: Fixed rare replication bug that normally only manifested when
using compound file format. (yonik, Mark Miller)

* SOLR-4974: Outgrowth of SOLR-4960 that includes transient cores and pending cores
(Erick Erickson)

* SOLR-3369: shards.tolerant=true is broken for group queries

(Russell Black, Martijn van Groningen, Jabouille jean Charles, Ryan McKinley via
shalin)

* SOLR-4452: Hunspell stemmer should not merge duplicate dictionary entries

(janhoy)

* SOLR-5000: ManagedIndexSchema doesn't persist uniqueKey tag after calling

addFields
method. (Jun Ohtani, Steve Rowe)

* SOLR-4982: Creating a core while referencing system properties looks like it

loses files
Actually, instanceDir, config, dataDir and schema are not dereferenced properly
when creating cores that reference sys vars (e.g. &dataDir=${dir}). In the
dataDir
case in particular this leads to the index being put in a directory literally
named
${dir} but on restart the sysvar will be properly dereferenced.

* SOLR-4788: Multiple Entities DIH delta import: dataimporter.

[entityName].last_index_time
is empty. (chakming wong, James Dyer via shalin)

* SOLR-4978: Time is stripped from datetime column when imported into Solr date
field
if convertType=true. (Bill Au, shalin)

* SOLR-5019: spurious ConcurrentModificationException when spell check component

was in use with filters. (yonik)

* SOLR-5018: The Overseer should avoid publishing the state for collections that do
not
exist under the /collections zk node. (Mark Miller)

* SOLR-5028,SOLR-5029: ShardHandlerFactory was not being created properly when

 using new-style solr.xml, and was not being persisted properly when using
old-style. (Tomás Fernández Löbbe, Ryan Ernst, Alan Woodward)

* SOLR-4997: The splitshard api doesn't call commit on new sub shards before
switching shard states. Multiple bugs related to sub shard recovery and
replication are also fixed. (shalin)

* SOLR-5034: A facet.query that parses or analyzes down to a null Query would

throw a NPE. Fixed. (David Smiley)

* SOLR-5039: Admin/Schema Browser displays -1 for term counts for multiValued

fields.

* SOLR-5037: The CSV loader now accepts field names that are not in the schema.
(gsingers, ehatcher, Steve Rowe)

* SOLR-4791: solr.xml sharedLib does not work in 4.3.0 (Ryan Ernst, Jan Høydahl via
Erick Erickson)

Optimizations
----------------------

* SOLR-4923: Commit to all nodes in a collection in parallel rather than locally

and
then to all other nodes. (hossman, Ricardo Merizalde, Mark Miller)

* SOLR-3838: Admin UI - Multiple filter queries are not supported in Query UI

(steffkes)

* SOLR-4719 : Admin UI - Default to wt=json on Query-Screen (steffkes)

* SOLR-4611: Admin UI - Analysis-Urls with empty parameters create empty result

table
(steffkes)

* SOLR-4955: Admin UI - Show address bar on top for Schema + Config (steffkes)

* SOLR-4412: New parameter langid.lcmap to map detected language code to be placed

in "language" field (janhoy)

* SOLR-4815: Admin-UI - DIH: Let "commit" be checked by default (steffkes)

* SOLR-5002: optimize numDocs(Query,DocSet) when filterCache is null (Robert Muir)

* SOLR-5012: optimize search with filter when filterCache is null (Robert Muir)

Other Changes
----------------------

* SOLR-4737: Update Guava to 14.0.1 (Mark Miller)

* SOLR-2079: Add option to pass HttpServletRequest in the SolrQueryRequest context

map.
(Tomás Fernández Löbbe via Robert Muir)

* SOLR-4738: Update Jetty to 8.1.10.v20130312 (Mark Miller, Robert Muir)

* SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore
management. (Mark Miller)
* SOLR-4547: Move logging of filenames on commit from INFO to DEBUG.
(Shawn Heisey, hossman)

* SOLR-4757: Change the example to use the new solr.xml format and core
discovery by directory structure. (Mark Miller)

* SOLR-4759: Velocity (/browse) template cosmetic cleanup.

(Mark Bennett, ehatcher)

* SOLR-4778: LogWatcher init code moved out of CoreContainer (Alan Woodward)

* SOLR-4784: Make class LuceneQParser public (janhoy)

* SOLR-4448: Allow the solr internal load balancer to be more easily pluggable.
(Philip Hoy via Robert Muir)

* SOLR-4224: Refactor JavaBinCodec input stream definition to enhance reuse.

(phunt via Mark Miller)

* SOLR-4931: SolrDeletionPolicy onInit and onCommit methods changed to override

exact signatures (with generics) from IndexDeletionPolicy (shalin)

* SOLR-4942: test improvements to randomize use of compound files (hossman)

* SOLR-4966: CSS, JS and other files in webapp without license (uschindler,

steffkes)

* SOLR-4986: Upgrade to Tika 1.4 (Markus Jelsma via janhoy)

* SOLR-4948, SOLR-5009: Tidied up CoreContainer construction logic.

(Alan Woodward, Uwe Schindler, Steve Rowe)

* LUCENE-5107: Properties files by Solr are now written in UTF-8 encoding,

Unicode is no longer escaped. Reading of legacy properties files with
\u escapes is still possible. (Uwe Schindler, Robert Muir)

================== 4.3.1 ==================

Versions of Major Components

---------------------
Apache Tika 1.3
Carrot2 3.6.2
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Detailed Change List

----------------------

Bug Fixes
----------------------

* SOLR-4795: Sub shard leader should not accept any updates from parent after
it goes active (shalin)

* SOLR-4798: shard splitting does not respect the router for the collection
when executing the index split. One effect of this is that documents
may be placed in the wrong shard when the default compositeId router
 is used in conjunction with IDs containing "!". (yonik)

* SOLR-4797: Shard splitting creates sub shards which have the wrong hash
range in cluster state. This happens when numShards is not a power of two
and router is compositeId. (shalin)

* SOLR-4806: Shard splitting does not abort if WaitForState times out (shalin)

* SOLR-4807: The zkcli script now works with log4j. The zkcli.bat script
was broken on Windows in 4.3.0, now it works. (Shawn Heisey)

* SOLR-4813: Fix SynonymFilterFactory to allow init parameters for

tokenizer factory used when parsing synonyms file. (Shingo Sasaki, hossman)

* SOLR-4829: Fix transaction log leaks (a failure to clean up some old logs)
on a shard leader, or when unexpected exceptions are thrown during log
recovery. (Steven Bower, Mark Miller, yonik)

* SOLR-4751: Fix replication problem of files in sub directory of conf directory.

(Minoru Osuka via Koji)

* SOLR-4741: Deleting a collection should set DELETE_DATA_DIR to true.

(Mark Miller)

* SOLR-4752: There are some minor bugs in the Collections API parameter
validation. (Mark Miller)

* SOLR-4563: RSS DIH-example not working (janhoy)

* SOLR-4796: zkcli.sh should honor JAVA_HOME (Roman Shaposhnik via Mark Miller)

* SOLR-4734: Leader election fails with an NPE if there is no UpdateLog.

(Mark Miller, Alexander Eibner)

* SOLR-4868: Setting the log level for the log4j root category results in
adding a new category, the empty string. (Shawn Heisey)

* SOLR-4855: DistributedUpdateProcessor doesn't check for peer sync requests

(shalin)

* SOLR-4867: Admin UI - setting loglevel on root throws RangeError (steffkes)

* SOLR-4870: RecentUpdates.update() does not increment numUpdates loop counter

(Alexey Kudinov via shalin)

* SOLR-4877, LUCENE-5023: Removed SolrIndexSearcher#getDocSetNC()'s special

case for handling TermQuery to prevent NullPointerException if reader does
not have fields. (Bao Yang Yang, Uwe Schindler)

* SOLR-4881: Fix DocumentAnalysisRequestHandler to correctly use

EmptyEntityResolver to prevent loading of external entities like
UpdateRequestHandler does. (Hossman, Uwe Schindler)

* SOLR-4858: SolrCore reloading was broken when the UpdateLog

was enabled. (Hossman, Anshum Gupta, Alexey Serba, Mark Miller, yonik)

* SOLR-4853: Fixed SolrJettyTestBase so it may be reused by end users

(hossman)
* SOLR-4744: Update failure on sub shard is not propagated to clients by parent
shard (Anshum Gupta, yonik, shalin)

Other Changes
----------------------

* SOLR-4760: Include core name in logs when loading schema.

(Shawn Heisey)

================== 4.3.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.3
Carrot2 3.6.2
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Upgrading from Solr 4.2.0

----------------------

* In the schema REST API, the output path for copyFields and dynamicFields
has been changed from all lowercase "copyfields" and "dynamicfields" to
camelCase "copyFields" and "dynamicFields", respectively, to align with all
other schema REST API outputs, which use camelCase. The URL format remains
the same: all resource names are lowercase. See SOLR-4623 for details.

* Slf4j/logging jars are no longer included in the Solr webapp. All logging
jars are now in example/lib/ext. Changing logging impls is now as easy as
updating the jars in this folder with those necessary for the logging impl
you would like. If you are using another webapp container, these jars will
need to go in the corresponding location for that container.
In conjunction, the dist-excl-slf4j and dist-war-excl-slf4 build targets
have been removed since they are redundant. See the Slf4j documentation,
SOLR-3706, and SOLR-4651 for more details.

* The hardcoded SolrCloud defaults for 'hostContext="solr"' and

'hostPort="8983"' have been deprecated and will be removed in Solr 5.0.
Existing solr.xml files that do not have these options explicitly specified
should be updated accordingly. See SOLR-4622 for more details.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-4648 PreAnalyzedUpdateProcessorFactory allows using the functionality

of PreAnalyzedField with other field types. See javadoc for details and
examples. (Andrzej Bialecki)

* SOLR-4623: Provide REST API read access to all elements of the live schema.
Add a REST API request to return the entire live schema, in JSON, XML, and
schema.xml formats. Move REST API methods from package org.apache.solr.rest
to org.apache.solr.rest.schema, and rename base functionality REST API
classes to remove the current schema focus, to prepare for other non-schema
REST APIs. Change output path for copyFields and dynamicFields from
 "copyfields" and "dynamicfields" (all lowercase) to "copyFields" and
"dynamicFields", respectively, to align with all other REST API outputs, which
use camelCase.
(Steve Rowe)

* SOLR-4658: In preparation for REST API requests that can modify the schema,
a "managed schema" is introduced.
Add '<schemaFactory class="ManagedSchemaFactory" mutable="true"/>' to
solrconfig.xml
in order to use it, and to enable schema modifications via REST API requests.
(Steve Rowe, Robert Muir)

* SOLR-4656: Added two new highlight parameters, hl.maxMultiValuedToMatch and

hl.maxMultiValuedToExamine. maxMultiValuedToMatch stops looking for snippets
after
finding the specified number of matches, no matter how far into the multivalued
field
you've gone. maxMultiValuedToExamine stops looking for matches after the
specified
number of multiValued entries have been examined. If both are specified, the
limit
hit first stops the loop. Also this patch cuts down on the copying of the
document
entries during highlighting. These optimizations are probably unnoticeable unless
there are a large number of entries in the multiValued field. Conspicuously, this
will
prevent the "best" match from being found if it appears later in the MV list than
the
cutoff specified by either of these params. (Erick Erickson)

* SOLR-4675: Improve PostingsSolrHighlighter to support per-field/query-time

overrides
and add additional configuration parameters. See the javadocs for more details
and
examples. (Robert Muir)

* SOLR-3755: A new collections api to add additional shards dynamically by

splitting
existing shards. (yonik, Anshum Gupta, shalin)

* SOLR-4530: DIH: Provide configuration to use Tika's IdentityHtmlMapper

(Alexandre Rafalovitch via shalin)

* SOLR-4662: Discover SolrCores by directory structure rather than defining them

in solr.xml. Also, change the format of solr.xml to be closer to that of
solrconfig.xml.
This version of Solr will ship the example in the old style, but you can manually
try the new style. Solr 4.4 will ship with the new style, and Solr 5.0 will
remove
support for the old style. (Erick Erickson, Mark Miller)
Additional Work:
- SOLR-4347: Ensure that newly-created cores via Admin handler are persisted in
solr.xml
(Erick Erickson)
- SOLR-1905: Cores created by the admin request handler should be persisted to
solr.xml.
Also fixed a problem whereby properties like solr.solr.datadir would be
persisted
to solr.xml. Also, cores that didn't happen to be loaded were not persisted.
 (Erick Erickson)

* SOLR-4717/SOLR-1351: SimpleFacets now work with localParams allowing faceting on

the
same field multiple ways (ryan, Uri Boness)

* SOLR-4671: CSVResponseWriter now supports pseudo fields. (ryan, nihed mbarek)

* SOLR-4358: HttpSolrServer sends the stream name and exposes 'useMultiPartPost'

(Karl Wright via ryan)

Bug Fixes
----------------------

* SOLR-4543: setting shardHandlerFactory in solr.xml/solr.properties does not work.

(Ryan Ernst, Robert Muir via Erick Erickson)

* SOLR-4634: Fix scripting engine tests to work with Java 8's "Nashorn" Javascript
implementation. (Uwe Schindler)

* SOLR-4636: If opening a reader fails for some reason when opening a

SolrIndexSearcher,
a Directory can be left unreleased. (Mark Miller)

* SOLR-4405: Admin UI - admin-extra files are not rendered into the core-menu
(steffkes)

* SOLR-3956: Fixed group.facet=true to work with negative facet.limit

(Chris van der Merwe, hossman)

* SOLR-4650: copyField doesn't work with source globs that don't match any
explicit or dynamic fields. This regression was introduced in Solr 4.2.
(Daniel Collins, Steve Rowe)

* SOLR-4641: Schema now throws exception on illegal field parameters. (Robert

Muir)

* SOLR-3758: Fixed SpellCheckComponent to work consistently with distributed

grouping
(James Dyer)

* SOLR-4652: Fix broken behavior with shared libraries in resource loader for
solr.xml plugins. (Ryan Ernst, Robert Muir, Uwe Schindler)

* SOLR-4664: ZkStateReader should update aliases on construction.

(Mark Miller, Elodie Sannier)

* SOLR-4682: CoreAdminRequest.mergeIndexes can not merge multiple cores or

indexDirs.
(Jason.D.Cao via shalin)

* SOLR-4581: When faceting on numeric fields in Solr 4.2, negative values

(constraints)
were sorted incorrectly. (Alexander Buhr, shalin, yonik)

* SOLR-4699: The System admin handler should not assume a file system based data
directory
location. (Mark Miller)
* SOLR-4695: Fix core admin SPLIT action to be useful with non-cloud setups
(shalin)

* SOLR-4680: Correct example spellcheck configuration's queryAnalyzerFieldType and

use "text" field instead of narrower "name" field (ehatcher, Mark Bennett)

* SOLR-4702: Fix example /browse "Did you mean?" suggestion feature. (ehatcher,
Mark Bennett)

* SOLR-4710: You cannot delete a collection fully from ZooKeeper unless all nodes
are up and
functioning correctly. (Mark Miller)

* SOLR-4487: SolrExceptions thrown by HttpSolrServer will now contain the

proper HTTP status code returned by the remote server, even if that status
code is not something Solr itself returned -- eg: from the Servlet Container,
or an intermediate HTTP Proxy (hossman)

* SOLR-4661: Admin UI Replication details now correctly displays the current

replicable generation/version of the master. (hossman)

* SOLR-4716,SOLR-4584: SolrCloud request proxying does not work on Tomcat and

perhaps other non Jetty containers. (Po Rui, Yago Riveiro via Mark Miller)

* SOLR-4746: Distributed grouping used a NamedList instead of a SimpleOrderedMap

for the top level group commands, causing output formatting differences
compared to non-distributed grouping. (yonik)

* SOLR-4705: Fixed bug causing NPE when querying a single replica in SolrCloud
using the shards param (Raintung Li, hossman)

* SOLR-4729: LukeRequestHandler: Using a dynamic copyField source that is

not also a dynamic field triggers error message 'undefined field: "(glob)"'.
(Adam Hahn, hossman, Steve Rowe)

Optimizations
----------------------

Other Changes
----------------------

* SOLR-4653: Solr configuration should log inaccessible/ non-existent relative

paths in lib
dir=... (Dawid Weiss)

* SOLR-4317: SolrTestCaseJ4: Can't avoid "collection1" convention (Tricia Jenkins,

via Erick Erickson)

* SOLR-4571: SolrZkClient#setData should return Stat object. (Mark Miller)

* SOLR-4603: CachingDirectoryFactory should use an IdentityHashMap for

byDirectoryCache. (Mark Miller)

* SOLR-4544: Refactor HttpShardHandlerFactory so load-balancing logic can be

customized.
(Ryan Ernst via Robert Muir)

* SOLR-4607: Use noggit 0.5 release jar rather than a forked copy. (Yonik Seeley,
Robert Muir)

* SOLR-3706: Ship setup to log with log4j. (ryan, Mark Miller)

* SOLR-4651: Remove dist-excl-slf4j build target. (Shawn Heisey)

* SOLR-4622: The hardcoded SolrCloud defaults for 'hostContext="solr"' and

'hostPort="8983"' have been deprecated and will be removed in Solr 5.0.
Existing solr.xml files that do not have these options explicitly specified
should be updated accordingly. (hossman)

* SOLR-4672: Requests attempting to use SolrCores which had init failures

(that would be reported by CoreAdmin STATUS requests) now result in 500
error responses with the details about the init failure, instead of 404
error responses. (hossman)

* SOLR-4730: Make the wiki link more prominent in the release documentation.
(Uri Laserson via Robert Muir)

================== 4.2.1 ==================

Versions of Major Components

---------------------
Apache Tika 1.3
Carrot2 3.6.2
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Detailed Change List

----------------------

Bug Fixes
----------------------

* SOLR-4567: copyField source glob matching explicit field(s) stopped working

in Solr 4.2. (Alexandre Rafalovitch, Steve Rowe)

* SOLR-4475: Fix various places that still assume File based paths even when
not using a file based DirectoryFactory. (Mark Miller)

* SOLR-4551: CachingDirectoryFactory needs to create CacheEntry's with the

fullpath not path. (Mark Miller)

* SOLR-4555: When forceNew is used with CachingDirectoryFactory#get, the old

CacheValue should give up its path as it will be used by a new Directory
instance. (Mark Miller)

* SOLR-4578: CoreAdminHandler#handleCreateAction gets a SolrCore and does not

close it in SolrCloud mode when a core with the same name already exists.
(Mark Miller)

* SOLR-4574: The Collections API will silently return success on an unknown

ACTION parameter. (Mark Miller)

* SOLR-4576: Collections API validation errors should cause an exception on

clients and otherwise act as validation errors with the Core Admin API.
(Mark Miller)
* SOLR-4577: The collections API should return responses (success or failure)
for each node it attempts to work with. (Mark Miller)

* SOLR-4568: The lastPublished state check before becoming a leader is not

working correctly. (Mark Miller)

* SOLR-4570: Even if an explicit shard id is used, ZkController#preRegister

should still wait to see the shard id in its current ClusterState.
(Mark Miller)

* SOLR-4585: The Collections API validates numShards with < 0 but should use
<= 0. (Mark Miller)

* SOLR-4592: DefaultSolrCoreState#doRecovery needs to check the CoreContainer

shutdown flag inside the recoveryLock sync block. (Mark Miller)

* SOLR-4595: CachingDirectoryFactory#close can throw a concurrent

modification exception. (Mark Miller)

* SOLR-4573: Accessing Admin UI files in SolrCloud mode logs warnings.

(Mark Miller, Phil John)

* SOLR-4594: StandardDirectoryFactory#remove accesses byDirectoryCache

without a lock. (Mark Miller)

* SOLR-4597: CachingDirectoryFactory#remove should not attempt to empty/remove

the index right away but flag for removal after close. (Mark Miller)

* SOLR-4598: The Core Admin unload command's option 'deleteDataDir', should use
the DirectoryFactory API to remove the data dir. (Mark Miller)

* SOLR-4599: CachingDirectoryFactory calls close(Directory) on forceNew if the

Directory has a refCnt of 0, but it should call closeDirectory(CacheValue).
(Mark Miller)

* SOLR-4602: ZkController#unregister should cancel its election participation

before asking the Overseer to delete the SolrCore information. (Mark Miller)

* SOLR-4601: A Collection that is only partially created and then deleted will
leave pre allocated shard information in ZooKeeper. (Mark Miller)

* SOLR-4604: UpdateLog#init is over called on SolrCore#reload. (Mark Miller)

* SOLR-4605: Rollback does not work correctly. (Mark S, Mark Miller)

* SOLR-4609: The Collections API should only send the reload command to ACTIVE
cores. (Mark Miller)

* SOLR-4297: Atomic update request containing null=true sets all subsequent

fields to null (Ben Pennell, Rob, shalin)

* SOLR-4371: Admin UI - Analysis Screen shows empty result (steffkes)

* SOLR-4318: NPE encountered with querying with wildcards on a field that uses
the DefaultAnalyzer (i.e. no analysis chain defined). (Erick Erickson)

* SOLR-4361: DataImportHandler would throw UnsupportedOperationException if

handler-level parameters were specified containing periods in the name
 (James Dyer)

* SOLR-4538: Date Math expressions were being truncated to 32 characters

when used in field:value queries in the lucene QParser. (hossman, yonik)

* SOLR-4617: SolrCore#reload needs to pass the deletion policy to the next

SolrCore through its constructor rather than setting a field after.
(Mark Miller)

* SOLR-4589: Fixed CPU spikes and poor performance in lazy field loading
of multivalued fields. (hossman)

* SOLR-4608: Update Log replay and PeerSync replay should use the default
processor chain to update the index. (Ludovic Boutros, yonik)

* SOLR-4625: The solr (lucene syntax) query parser lost top-level boost
values and top-level phrase slops on queries produced by nested
sub-parsers. (yonik)

* SOLR-4624: CachingDirectoryFactory does not need to support forceNew any

longer and it appears to be causing a missing close directory bug. forceNew
is no longer respected and will be removed in 4.3. (Mark Miller)

* SOLR-3819: Grouped faceting (group.facet=true) did not respect filter

exclusions. (Petter Remen, yonik)

* SOLR-4637: Replication can sometimes wait until shutdown or core unload until
removing some tmp directories. (Mark Miller)

* SOLR-4638: DefaultSolrCoreState#getIndexWriter(null) is a way to avoid

creating the IndexWriter earlier than necessary, but it's not
implemented quite right. (Mark Miller)

* SOLR-4640: CachingDirectoryFactory can fail to close directories in some race

conditions. (Mark Miller)

* SOLR-4642: QueryResultKey is not calculating the correct hashCode for filters.

(Joel Bernstein via Mark Miller)

Optimizations
----------------------

* SOLR-4569: waitForReplicasToComeUp should bail right away if it doesn't see the

expected slice in the clusterstate rather than waiting. (Mark Miller)

* SOLR-4311: Admin UI - Optimize Caching Behaviour (steffkes)

Other Changes
----------------------

* SOLR-4537: Clean up schema information REST API. (Steve Rowe)

* SOLR-4596: DistributedQueue should ensure its full path exists in the

constructor.
(Mark Miller)

================== 4.2.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.3
Carrot2 3.6.2
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Upgrading from Solr 4.1.0

----------------------

(No upgrade instructions yet)

Detailed Change List

----------------------

New Features
----------------------

* SOLR-4043: Add ability to get success/failure responses from Collections API.

(Raintung Li, Mark Miller)

* SOLR-2827: RegexpBoost Update Processor (janhoy)

* SOLR-4370: Allow configuring commitWithin to do hard commits.

(Mark Miller, Senthuran Sivananthan)

* SOLR-4451: SolrJ, and SolrCloud internals, now use SystemDefaultHttpClient

under the covers -- allowing many HTTP connection related properties to be
controlled via 'standard' java system properties. (hossman)

* SOLR-3855, SOLR-4490: Doc values support. (Adrien Grand, Robert Muir)

* SOLR-4417: Reopen the IndexWriter on SolrCore reload. (Mark Miller)

* SOLR-4477: Add support for queries (match-only) against docvalues fields.

(Robert Muir)

* SOLR-4488: Return slave replication details for a master if the master has
also acted like a slave. (Mark Miller)

* SOLR-4498: Add list command to ZkCLI that prints out the contents of
ZooKeeper. (Roman Shaposhnik via Mark Miller)

* SOLR-4481: SwitchQParserPlugin registered by default as 'switch' using

syntax: {!switch case=XXX case.foo=YYY case.bar=ZZZ default=QQQ}foo
(hossman)

* SOLR-4078: Allow custom naming of SolrCloud nodes so that a new host:port

combination can take over for a previous shard. (Mark Miller)

* SOLR-4210: Requests to a Collection that does not exist on the receiving node
should be proxied to a suitable node. (Mark Miller, Po Rui, yonik)

* SOLR-1365: New SweetSpotSimilarityFactory allows customizable TF/IDF based

Similarity when you know the optimal "Sweet Spot" of values for the field
length and TF scoring factors. (hossman)

* SOLR-4138: CurrencyField fields can now be used in a ValueSources to

get the "raw" value (using the default number of fractional digits) in
 the default currency of the field type. There is also a new
currency(field,[CODE]) function for generating a ValueSource of the
"natural" value, converted to an optionally specified currency to
override the default for the field type.
(hossman)

* SOLR-4503: Add REST API methods, via Restlet integration, for reading schema
elements, at /schema/fields/, /schema/dynamicfields/, /schema/fieldtypes/,
and /schema/copyfields/. (Steve Rowe)

Bug Fixes
----------------------

* SOLR-2850: Do not refine facets when minCount == 1

(Matt Smith, lundgren via Adrien Grand)

* SOLR-4309: /browse: Improve JQuery autosuggest behavior (janhoy)

* SOLR-4330: group.sort is ignored when using group.truncate and ex/tag

local params together (koji)

* SOLR-4321: Collections API will sometimes use a node more than once, even
when more unused nodes are available.
(Eric Falcao, Brett Hoerner, Mark Miller)

* SOLR-4345 : Solr Admin UI doesn't work in IE 10 (steffkes)

* SOLR-4349 : Admin UI - Query Interface does not work in IE

(steffkes)

* SOLR-4359: The RecentUpdates#update method should treat a problem reading the

next record the same as a problem parsing the record - log the exception and
break. (Mark Miller)

* SOLR-4225: Term info page under schema browser shows incorrect count of terms
(steffkes)

* SOLR-3926: Solr should support better way of finding active sorts (Eirik Lygre
via
Erick Erickson)

* SOLR-4342: Fix DataImportHandler stats to be a proper Map (hossman)

* SOLR-3967: langid.enforceSchema option checks source field instead of target

field (janhoy)

* SOLR-4380: Replicate after startup option would not replicate until the
IndexWriter was lazily opened. (Mark Miller, Gregg Donovan)

* SOLR-4400: Deadlock can occur in a rare race between committing and

closing a SolrIndexWriter. (Erick Erickson, Mark Miller)

* SOLR-3655: A restarted node can briefly appear live and active before it really
is in some cases. (Mark Miller)

* SOLR-4426: NRTCachingDirectoryFactory does not initialize maxCachedMB and

maxMergeSizeMB
if <directoryFactory> is not present in solrconfig.xml (Jack Krupansky via
shalin)
* SOLR-4463: Fix SolrCoreState reference counting. (Mark Miller)

* SOLR-4459: The Replication 'index move' rather than copy optimization doesn't
kick in when using NRTCachingDirectory or the rate limiting feature.
(Mark Miller)

* SOLR-4421,SOLR-4165: On CoreContainer shutdown, all SolrCores should publish

their
state as DOWN. (Mark Miller, Markus Jelsma)

* SOLR-4467: Ephemeral directory implementations may not recover correctly

because the code to clear the tlog files on startup is off. (Mark Miller)

* SOLR-4413: Fix SolrCore#getIndexDir() to return the current index directory.

(Gregg Donovan, Mark Miller)

* SOLR-4469: A new IndexWriter must be opened on SolrCore reload when the index
directory has changed and the previous SolrCore's state should not be
propagated. (Mark Miller, Gregg Donovan)

* SOLR-4471: Replication occurs even when a slave is already up to date.

(Mark Miller, Andre Charton)

* SOLR-4484: ReplicationHandler#loadReplicationProperties still uses Files

rather than the Directory to try and read the replication properties files.
(Mark Miller)

* SOLR-4352: /browse pagination now supports and preserves sort context

(Eric Spiegelberg, Erik Hatcher)

* LUCENE-4796, SOLR-4373: Fix concurrency issue in NamedSPILoader and

AnalysisSPILoader when doing concurrent core loads in multicore
Solr configs. (Uwe Schindler, Hossman)

* SOLR-4504: Fixed CurrencyField range queries to correctly exclude

documents w/o values (hossman)

* SOLR-4480: A trailing + or - caused the edismax parser to throw

an exception. (Fiona Tay, Jan Høydahl, yonik)

* SOLR-4507: The Cloud tab does not show up in the Admin UI if you
set zkHost in solr.xml. (Alfonso Presa, Mark Miller)

* SOLR-4505: Possible deadlock around SolrCoreState update lock.

(Erick Erickson, Mark Miller)

* SOLR-4511: When a new index is replicated into place, we need

to update the most recent replicatable index point without
doing a commit. This is important for repeater use cases, as
well as when nodes may switch master/slave roles.
(Mark Miller, Raúl Grande)

* SOLR-4515: CurrencyField's OpenExchangeRatesOrgProvider now requires

a ratesFileLocation init param, since the previous global default
no longer works (hossman)

* SOLR-4518: Improved CurrencyField error messages when attempting to

use a Currency that is not supported by the current JVM. (hossman)
* SOLR-3798: Fix copyField implementation in IndexSchema to handle
dynamic field references that aren't string-equal to the name of
the referenced dynamic field. (Steve Rowe)

* SOLR-4497: Collection Aliasing. (Mark Miller)

Optimizations
----------------------

* SOLR-4339: Admin UI - Display Field-Flags on Schema-Browser

(steffkes)

* SOLR-4340: Admin UI - Analysis's Button Spinner goes wild

(steffkes)

* SOLR-4341: Admin UI - Plugins/Stats Page contains loooong

Values which result in horizontal Scrollbar (steffkes)

* SOLR-3915: Color Legend for Cloud UI (steffkes)

* SOLR-4306: Utilize indexInfo=false when gathering core names in UI

(steffkes)

* SOLR-4284: Admin UI - make core list scrollable separate from the rest of
the UI (steffkes)

* SOLR-4364: Admin UI - Locale based number formatting (steffkes)

* SOLR-4521: Stop using the 'force' option for recovery replication. This
will keep some less common unnecessary replications from happening.
(Mark Miller, Simon Scofield)

* SOLR-4529: Improve Admin UI Dashboard legibility (Felix Buenemann via

steffkes)

* SOLR-4526: Admin UI depends on optional system info (Felix Buenemann via

steffkes)

Other Changes
----------------------

* SOLR-4259: Carrot2 dependency should be declared on the mini version, not the
core.
(Dawid Weiss).

* SOLR-4348: Make the lock type configurable by system property by default.

(Mark Miller)

* SOLR-4353: Renamed example jetty context file to reduce confusion (hossman)

* SOLR-4384: Make post.jar report timing information (Upayavira via janhoy)

* SOLR-4415: Add 'state' to shards (default to 'active') and read/write them to

ZooKeeper (Anshum Gupta via shalin)

* SOLR-4394: Tests and example configs demonstrating SSL with both server
and client certs (hossman)
* SOLR-3060: SurroundQParserPlugin highlighting tests
(Ahmet Arslan via hossman)

* SOLR-2470: Added more tests for VelocityResponseWriter

* SOLR-4471: Improve and clean up TestReplicationHandler.

(Amit Nithian via Mark Miller)

* SOLR-3843: Include lucene codecs jar and enable per-field postings and docvalues
support in the schema.xml (Robert Muir, Steve Rowe)

* SOLR-4511: Add new test for 'repeater' replication node. (Mark Miller)

* SOLR-4458: Sort directions (asc, desc) are now case insensitive

(Shawn Heisey via hossman)

* SOLR-2996: A bare * without a field specification is treated as *:*

by the lucene and edismax query parsers.
(hossman, Jan Høydahl, Alan Woodward, yonik)

* SOLR-4416: Upgrade to Tika 1.3. (Markus Jelsma via Mark Miller)

* SOLR-4200: Reduce INFO level logging from CachingDirectoryFactory

(Shawn Heisey via hossman)

================== 4.1.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.2
Carrot2 3.6.2
Velocity 1.7 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.4.5

Upgrading from Solr 4.0.0

----------------------

Custom java parsing plugins need to migrate from throwing the internal
ParseException to throwing SyntaxError.

BaseDistributedSearchTestCase now randomizes the servlet context it uses when

creating Jetty instances. Subclasses that assume a hard coded context of
"/solr" should either be fixed to use the "String context" variable, or should
take advantage of the new BaseDistributedSearchTestCase(String) constructor
to explicitly specify a fixed servlet context path. See SOLR-4136 for details.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-2255: Enhanced pivot faceting to use local-params in the same way that
regular field value faceting can. This means support for excluding a filter
query, using a different output key, and specifying 'threads' to do
facet.method=fcs concurrently. PivotFacetHelper now extends SimpleFacet and
the getFacetImplementation() extension hook was removed. (dsmiley)
* SOLR-3897: A highlighter parameter "hl.preserveMulti" to return all of the
values of a multiValued field in their original order when highlighting.
(Joel Bernstein via yonik)

* SOLR-3929: Support configuring IndexWriter max thread count in solrconfig.

(phunt via Mark Miller)

* SOLR-3906: Add support for AnalyzingSuggester (LUCENE-3842), where the

underlying analyzed form used for suggestions is separate from the returned
text. (Robert Muir)

* SOLR-3985: ExternalFileField caches can be reloaded on firstSearcher/

newSearcher events using the ExternalFileFieldReloader (Alan Woodward)

* SOLR-3911: Make Directory and DirectoryFactory first class so that the majority
of Solr's features work with any custom implementations. (Mark Miller)
Additional Work:
- SOLR-4032: Files larger than an internal buffer size fail to replicate.
(Mark Miller, Markus Jelsma)
- SOLR-4033: Consistently use the solrconfig.xml lockType everywhere.
(Mark Miller, Markus Jelsma)
- SOLR-4144: Replication using too much RAM. (yonik, Markus Jelsma)
- SOLR-4187: NPE on Directory release (Mark Miller, Markus Jelsma)

* SOLR-4051: Add <propertyWriter /> element to DIH's data-config.xml file,

allowing the user to specify the location, filename and Locale for
the "data-config.properties" file. Alternatively, users can specify their
own property writer implementation for greater control. This new configuration
element is optional, and defaults mimic prior behavior. The one exception is
that the "root" locale is default. Previously it was the machine's default
locale.
(James Dyer)

* SOLR-4084: Add FuzzyLookupFactory, which is like AnalyzingSuggester except that

it can tolerate typos in the input. (Areek Zillur via Robert Muir)

* SOLR-4088: New and improved auto host detection strategy for SolrCloud.
(Raintung Li via Mark Miller)

* SOLR-3970: SystemInfoHandler now exposes more details about the

JRE/VM/Java version in use. (hossman)

* SOLR-4101: Add support for storing term offsets in the index via a
'storeOffsetsWithPositions' flag on field definitions in the schema.
(Tom Winch, Alan Woodward)

* SOLR-4093: Solr QParsers may now be directly invoked in the lucene

query syntax without the _query_ magic field hack.
Example: foo AND {!term f=myfield v=$qq}
(yonik)

* SOLR-4087: Add MAX_DOC_FREQ option to MoreLikeThis.

(Andrew Janowczyk via Mark Miller)

* SOLR-4114: Allow creating more than one shard per instance with the
Collection API. (Per Steffensen, Mark Miller)

* SOLR-3531: Allowing configuring maxMergeSizeMB and maxCachedMB when

 using NRTCachingDirectoryFactory. (Andy Laird via Mark Miller)

* SOLR-4118: Fix replicationFactor to align with industry usage.

replicationFactor now means the total number of copies
of a document stored in the collection (or the total number of
physical indexes for a single logical slice of the collection).
For example if replicationFactor=3 then for a given shard there
will be a total of 3 replicas (one of which will normally be
designated as the leader.) (yonik)

* SOLR-4124: You should be able to set the update log directory with the
CoreAdmin API the same way as the data directory. (Mark Miller)

* SOLR-4028: When using ZK chroot, it would be nice if Solr would create the
initial path when it doesn't exist. (Tomás Fernández Löbbe via Mark Miller)

* SOLR-3948: Calculate/display deleted documents in admin interface.

(Shawn Heisey via Mark Miller)

* SOLR-4030: Allow rate limiting Directory IO based on the IO context.

(Mark Miller, Radim Kolar)

* SOLR-4166: LBHttpSolrServer ignores ResponseParser passed in constructor.

(Steve Molloy via Mark Miller)

* SOLR-4140: Allow access to the collections API through CloudSolrServer

without referencing an existing collection. (Per Steffensen via Mark Miller)

* SOLR-788: Distributed search support for MLT.

(Matthew Woytowitz, Mike Anderson, Jamie Johnson, Mark Miller)

* SOLR-4120: Collection API: Support for specifying a list of Solr addresses to

spread a new collection across. (Per Steffensen via Mark Miller)

* SOLR-4110: Configurable Content-Type headers for PHPResponseWriters and

PHPSerializedResponseWriter. (Dominik Siebel via Mark Miller)

* SOLR-1028: The ability to specify "transient" and "loadOnStartup" as a new

properties of
<core> tags in solr.xml. Can specify "transientCacheSize" in the <cores> tag.
Together
these allow cores to be loaded only when needed and only transientCacheSize
transient
cores will be loaded at a time, the rest aged out on an LRU basis.

* SOLR-4246: When update.distrib is set to skip update processors before

the distributed update processor, always include the log update processor
so forwarded updates will still be logged. (yonik)

* SOLR-4230: The new Solr 4 spatial fields now work with the {!geofilt} and
{!bbox} query parsers. The score local-param works too. (David Smiley)

* SOLR-1972: Add extra statistics to RequestHandlers - 5 & 15-minute reqs/sec

rolling averages; median, 75th, 95th, 99th, 99.9th percentile request times
(Alan Woodward, Shawn Heisey, Adrien Grand, Uwe Schindler)

* SOLR-4271: Add support for PostingsHighlighter. (Robert Muir)

* SOLR-4255: The new Solr 4 spatial fields now have a 'filter' boolean local-param
 that can be set to false to not filter. It's useful when there is already a
spatial
filter query but you also need to sort or boost by distance. (David Smiley)

* SOLR-4265, SOLR-4283: Solr now parses request parameters (in URL or sent with
POST
using content-type application/x-www-form-urlencoded) in its dispatcher code. It
no
longer relies on special configuration settings in Tomcat or other web containers
to enable UTF-8 encoding, which is mandatory for correct Solr behaviour. Query
strings passed in via the URL need to be properly-%-escaped, UTF-8 encoded
bytes, otherwise Solr refuses to handle the request. The maximum length of
x-www-form-urlencoded POST parameters can now be configured through the
requestDispatcher/requestParsers/@formdataUploadLimitInKB setting in
solrconfig.xml (defaults to 2 MiB). Solr now works out of the box with
e.g. Tomcat, JBoss,... (Uwe Schindler, Dawid Weiss, Alex Rocher)

* SOLR-2201: DIH's "formatDate" function now supports a timezone as an optional

fourth parameter (James Dyer, Mark Waddle)

* SOLR-4302: New parameter 'indexInfo' (defaults to true) in CoreAdmin STATUS

command can be used to omit index specific information (Shahar Davidson via
shalin)

* SOLR-2592: Collection specific document routing. The "compositeId"

router is the default for collections with hash based routing (i.e. when
numShards=N is specified on collection creation). Documents with ids sharing
the same domain (prefix) will be routed to the same shard, allowing for
efficient querying.

Example:
The following two documents will be indexed to the same shard
since they share the same domain "customerB!".
<code>
{"id" : "customerB!doc1" [...] }
{"id" : "customerB!doc2" [...] }
</code>
At query time, one can specify a "shard.keys" parameter that lists what
shards the query should cover.
http://.../query?q=my_query&shard.keys=customerB!

Collections that do not specify numShards at collection creation time

use custom sharding and default to the "implicit" router. Document updates
received by a shard will be indexed to that shard, unless a "_shard_" parameter
or document field names a different shard.
(Michael Garski, Dan Rosher, yonik)

Optimizations
----------------------

* SOLR-3788: Admin Cores UI should redirect to newly created core details

(steffkes)

* SOLR-3895: XML and XSLT UpdateRequestHandler should not try to resolve

external entities. This improves speed of loading e.g. XSL-transformed
XHTML documents. (Martin Herfurt, uschindler, hossman)

* SOLR-3614: Fix XML parsing in XPathEntityProcessor to correctly expand

 named entities, but ignore external entities. (uschindler, hossman)

* SOLR-3734: Improve Schema-Browser Handling for CopyField using

dynamicField's (steffkes)

* SOLR-3941: The "commitOnLeader" part of distributed recovery can use

openSearcher=false. (Tomás Fernández Löbbe via Mark Miller)

* SOLR-4063: Allow CoreContainer to load multiple SolrCores in parallel rather

than just serially. (Mark Miller)

* SOLR-4199: When doing zk retries due to connection loss, rather than just
retrying for 2 minutes, retry in proportion to the session timeout.
(Mark Miller)

* SOLR-4262: Replication Icon on Dashboard does not reflect Master-/Slave-

State (steffkes)

* SOLR-4264: Missing Error-Screen on UI's Cloud-Page (steffkes)

* SOLR-4261: Percentage Infos on Dashboard have a fixed width (steffkes)

* SOLR-3851: create a new core/delete an existing core should also update

the main/left list of cores on the admin UI (steffkes)

* SOLR-3840: XML query response display is unreadable in Solr Admin Query UI

(steffkes)

* SOLR-3982: Admin UI: Various Dataimport Improvements (steffkes)

* SOLR-4296: Admin UI: Improve Dataimport Auto-Refresh (steffkes)

* SOLR-3458: Allow multiple Items to stay open on Plugins-Page (steffkes)

Bug Fixes
----------------------

* SOLR-4288: Improve logging for FileDataSource (basePath, relative

resources). (Dawid Weiss)

* SOLR-4007: Morfologik dictionaries not available in Solr field type

due to class loader lookup problems. (Lance Norskog, Dawid Weiss)

* SOLR-3560: Handle different types of Exception Messages for Logging UI

(steffkes)

* SOLR-3637: Commit Status at Core-Admin UI is always false (steffkes)

* SOLR-3917: Partial State on Schema-Browser UI is not defined for Dynamic

Fields & Types (steffkes)

* SOLR-3939: Consider a sync attempt from leader to replica that fails due
to 404 a success. (Mark Miller, Joel Bernstein)

* SOLR-3940: Rejoining the leader election incorrectly triggers the code path
for a fresh cluster start rather than fail over. (Mark Miller)

* SOLR-3961: Fixed error using LimitTokenCountFilterFactory

(Jack Krupansky, hossman)
* SOLR-3933: Distributed commits are not guaranteed to be ordered within a
request. (Mark Miller)

* SOLR-3939: An empty or just replicated index cannot become the leader of a

shard after a leader goes down. (Joel Bernstein, yonik, Mark Miller)

* SOLR-3971: A collection that is created with numShards=1 turns into a

numShards=2 collection after starting up a second core and not specifying
numShards. (Mark Miller)

* SOLR-3988: Fixed SolrTestCaseJ4.adoc(SolrInputDocument) to respect

field and document boosts (hossman)

* SOLR-3981: Fixed bug that resulted in document boosts being compounded in

<copyField/> destination fields. (hossman)

* SOLR-3920: Fix server list caching in CloudSolrServer when using more than one
collection list with the same instance. (Grzegorz Sobczyk, Mark Miller)

* SOLR-3938: prepareCommit command omits commitData causing a failure to trigger

replication to slaves. (yonik)

* SOLR-3992: QuerySenderListener doesn't populate document cache.

(Shotaro Kamio, yonik)

* SOLR-3995: Recovery may never finish on SolrCore shutdown if the last reference
to
a SolrCore is closed by the recovery process. (Mark Miller)

* SOLR-3998: Atomic update on uniqueKey field itself causes duplicate document.

(Eric Spencer, yonik)

* SOLR-4001: In CachingDirectoryFactory#close, if there are still refs for a

Directory outstanding, we need to wait for them to be released before closing.
(Mark Miller)

* SOLR-4005: If CoreContainer fails to register a created core, it should close it.

(Mark Miller)

* SOLR-4009: OverseerCollectionProcessor is not resilient to many error conditions

and can stop running on errors. (Raintung Li, milesli, Mark Miller)

* SOLR-4019: Log stack traces for 503/Service Unavailable SolrException if not

thrown by PingRequestHandler. Do not log exceptions if a user tries to view a
hidden file using ShowFileRequestHandler. (Tomás Fernández Löbbe via James Dyer)

* SOLR-3589: Edismax parser does not honor mm parameter if analyzer splits a token.
(Tom Burton-West, Robert Muir)

* SOLR-4031: Upgrade to Jetty 8.1.7 to fix a bug where in very rare occasions
the content of two concurrent requests get mixed up. (Per Steffensen, yonik)

* SOLR-4060: ReplicationHandler can try and do a snappull and open a new

IndexWriter
after shutdown has already occurred, leaving an IndexWriter that is not closed.
(Mark Miller)

* SOLR-4055: Fix a thread safety issue with the Collections API that could
 cause actions to be targeted at the wrong SolrCores.
(Raintung Li, Per Steffensen via Mark Miller)

* SOLR-3993: If multiple SolrCore's for a shard coexist on a node, on cluster

restart, leader election would stall until timeout, waiting to see all of
the replicas come up. (Mark Miller, Alexey Kudinov)

* SOLR-2045: Databases that require a commit to be issued before closing the

connection on a non-read-only database leak connections. Also expanded the
SqlEntityProcessor test to sometimes use Derby as well as HSQLDB (Derby is
one db affected by this bug). (Fenlor Sebastia, James Dyer)

* SOLR-4064: When there is an unexpected exception while trying to run the new
leader process, the SolrCore will not correctly rejoin the election.
(Po Rui via Mark Miller)

* SOLR-3989: SolrZkClient constructor dropped exception cause when throwing

a new RuntimeException. (Colin Bartolome, yonik)

* SOLR-4036: field aliases in fl should not cause properties of target field

to be used. (Martin Koch, yonik)

* SOLR-4003: The SolrZKClient clean method should not try and clear zk paths
that start with /zookeeper, as this can fail and stop the removal of
further nodes. (Mark Miller)

* SOLR-4076: SolrQueryParser should run fuzzy terms through

MultiTermAwareComponents to ensure that (for example) a fuzzy query of
foobar~2 is equivalent to FooBar~2 on a field that includes lowercasing.
(yonik)

* SOLR-4081: QueryParsing.toString, used during debugQuery=true, did not

correctly handle ExtendedQueries such as WrappedQuery
(used when cache=false), spatial queries, and frange queries.
(Eirik Lygre, yonik)

* SOLR-3959: Ensure the internal comma separator of poly fields is escaped

for CSVResponseWriter. (Areek Zillur via Robert Muir)

* SOLR-4075: A logical shard that has had all of its SolrCores unloaded should
be removed from the cluster state. (Mark Miller, Gilles Comeau)

* SOLR-4034: Check if a collection already exists before trying to create a

new one. (Po Rui, Mark Miller)

* SOLR-4097: Race can cause NPE in logging line on first cluster state update.
(Mark Miller)

* SOLR-4099: Allow the collection api work queue to make forward progress even
when its watcher is not fired for some reason. (Raintung Li via Mark Miller)

* SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters

(Nathan Visagan, hossman)

* SOLR-3842: DIH would not populate multivalued fields if the column name
derives from a resolved variable (James Dyer)

* SOLR-4117: Retrieving the size of the index may use the wrong index dir if
you are replicating.
 (Mark Miller, Markus Jelsma)

* SOLR-2890: Fixed a bug that prevented omitNorms and omitTermFreqAndPositions

options from being respected in some <fieldType/> declarations (hossman)

* SOLR-4159: When we are starting a shard from rest, a potential leader should
not consider its last published state when deciding if it can be the new
leader. (Mark Miller)

* SOLR-4158: When a core is registering in ZooKeeper it may not wait long

enough to find the leader due to how long the potential leader waits to see
replicas. (Mark Miller, Alain Rogister)

* SOLR-4162: ZkCli usage examples are not correct because the zkhost parameter
is not present and it is mandatory for all commands.
(Tomás Fernández Löbbe via Mark Miller)

* SOLR-4071: Validate that name is pass to Collections API create, and behave the
same way as on startup when collection.configName is not explicitly passed.
(Po Rui, Mark Miller)

* SOLR-4127: Added explicit error message if users attempt Atomic document

updates with either updateLog or DistribUpdateProcessor. (hossman)

* SOLR-4136: Fix SolrCloud behavior when using "hostContext" containing "_"

or"/" characters. This fix also makes SolrCloud more accepting of
hostContext values with leading/trailing slashes. (hossman)

* SOLR-4168: Ensure we are using the absolute latest index dir when getting
list of files for replication. (Mark Miller)

* SOLR-4171: CachingDirectoryFactory should not return any directories after it

has been closed. (Mark Miller)

* SOLR-4102: Fix UI javascript error if canonical hostname can not be resolved

(steffkes via hossman)

* SOLR-4178: ReplicationHandler should abort any current pulls and wait for
its executor to stop during core close. (Mark Miller)

* SOLR-3918: Fixed the 'dist-war-excl-slf4j' ant target to exclude all

slf4j jars, so that the resulting war is usable as is provided the servlet
container includes the correct slf4j api and impl jars.
(Shawn Heisey, hossman)

* SOLR-4198: OverseerCollectionProcessor should implement ClosableThread.

(Mark Miller)

* SOLR-4213: Directories that are not shutdown until DirectoryFactory#close

do not have close listeners called on them. (Mark Miller)

* SOLR-4134: Standard (XML) request writer cannot "set" multiple values into
multivalued field with partial updates. (Luis Cappa Banda, Will Butler, shalin)

* SOLR-3972: Fix ShowFileRequestHandler to not log a warning in the

(expected) situation of a file not found. (hossman)

* SOLR-4133: Cannot "set" field to null with partial updates when using the
standard RequestWriter. (Will Butler, shalin)
* SOLR-4223: "maxFormContentSize" in jetty.xml is not picked up by jetty 8
so set it via solr webapp context file. (shalin)

* SOLR-4175:SearchComponent chain can't contain two components of the

same class and use debugQuery. (Tomás Fernández Löbbe via ehatcher)

* SOLR-4244: When coming back from session expiration we should not wait for
the leader to see us in the down state if we are the node that must become
the leader. (Mark Miller)

* SOLR-4245: When a core is registering with ZooKeeper, the timeout to find the
leader in the cluster state is 30 seconds rather than leaderVoteWait + extra
time. (Mark Miller)

* SOLR-4238: Fix jetty example requestLog config (jm via hossman)

* SOLR-4251: Fix SynonymFilterFactory when an optional tokenizerFactory is

supplied.
(Chris Bleakley via rmuir)

* SOLR-4253: Misleading resource loading warning from Carrot2 clustering

component fixed (Stanisław Osiński)

* SOLR-4257: PeerSync updates and Log Replay updates should not wait for
a ZooKeeper connection in order to proceed. (yonik)

* SOLR-4045: SOLR admin page returns HTTP 404 on core names containing
a '.' (dot) (steffkes)

* SOLR-4176: analysis ui: javascript not properly handling URL decoding

of input (steffkes)

* SOLR-4079: Long core names break web gui appearance and functionality
(steffkes)

* SOLR-4263: Incorrect Link from Schema-Browser to Query From for Top-Terms

(steffkes)

* SOLR-3829: Admin UI Logging events broken if schema.xml defines a catch-all

dynamicField with type ignored (steffkes)

* SOLR-4275: Fix TrieTokenizer to no longer throw StringIndexOutOfBoundsException

in admin UI / AnalysisRequestHandler when you enter no number to tokenize.
(Uwe Schindler)

* SOLR-4279: Wrong exception message if _version_ field is multivalued (shalin)

* SOLR-4170: The 'backup' ReplicationHandler command can sometimes use a stale

index directory rather than the current one. (Mark Miller, Marcin Rzewucki)

* SOLR-3876: Solr Admin UI is completely dysfunctional on IE 9 (steffkes)

* SOLR-4112: Fixed DataImportHandler ZKAwarePropertiesWriter implementation so

import works fine with SolrCloud clusters (Deniz Durmus, James Dyer,
Erick Erickson, shalin)

* SOLR-4291: Harden the Overseer work queue thread loop. (Mark Miller)
* SOLR-3820: Solr Admin Query form is missing some edismax request parameters
(steffkes)

* SOLR-4217: post.jar no longer ignores -Dparams when -Durl is used.

(Alexandre Rafalovitch, ehatcher)

* SOLR-4303: On replication, if the generation of the master is lower than the

slave we need to force a full copy of the index. (Mark Miller, Gregg Donovan)

* SOLR-4266: HttpSolrServer does not release connection properly on exception

when no response parser is used. (Steve Molloy via Mark Miller)

* SOLR-2298: Updated JavaDoc for SolrDocument.addField and

SolrInputDocument.addField
to have more information on name and value parameters. (Siva Natarajan)

Other Changes
----------------------

* SOLR-4106: Javac/ ivy path warnings with morfologik fixed by

upgrading to Morfologik 1.5.5 (Robert Muir, Dawid Weiss)

* SOLR-3899: SolrCore should not log at warning level when the index directory
changes - it's an info event. (Tobias Bergman, Mark Miller)

* SOLR-3861: Refactor SolrCoreState so that it's managed by SolrCore.

(Mark Miller, hossman)

* SOLR-3966: Eliminate superfluous warning from LanguageIdentifierUpdateProcessor

(Markus Jelsma via hossman)

* SOLR-3932: SolrCmdDistributorTest either takes 3 seconds or 3 minutes.

(yonik, Mark Miller)

* SOLR-3856: New tests for SqlEntityProcessor/CachedSqlEntityProcessor

(James Dyer)

* SOLR-4067: ZkStateReader#getLeaderProps should not return props for a leader

that it does not think is live. (Mark Miller)

* SOLR-4086: DIH refactor of VariableResolver and Evaluator. VariableResolver

and each built-in Evaluator are separate concrete classes. DateFormatEvaluator
now defaults with the ROOT Locale. However, users may specify a different
Locale using an optional new third parameter. (James Dyer)

* SOLR-3602: Update ZooKeeper to 3.4.5 (Mark Miller)

* SOLR-4095: DIH NumberFormatTransformer & DateFormatTransformer default to the

ROOT Locale if none is specified. These previously used the machine's default.
(James Dyer)

* SOLR-4096: DIH FileDataSource & FieldReaderDataSource default to UTF-8 encoding

if none is specified. These previously used the machine's default.
(James Dyer)

* SOLR-1916: DIH to not use Lucene-forbidden Java APIs

(default encoding, locale, etc.) (James Dyer, Robert Muir)

* SOLR-4111: SpellCheckCollatorTest#testContextSensitiveCollate to test against

 both DirectSolrSpellChecker & IndexBasedSpellChecker
(Tomás Fernández Löbbe via James Dyer)

* SOLR-2141: Better test coverage for Evaluators (James Dyer)

* SOLR-4119: Update Guava to 13.0.1 (Mark Miller)

* SOLR-4074: Raise default ramBufferSizeMB to 100 from 32.

(yonik, Mark Miller)

* SOLR-4062: The update log location in solrconfig.xml should default to

${solr.ulog.dir} rather than ${solr.data.dir:} (Mark Miller)

* SOLR-4155: Upgrade Jetty to 8.1.8. (Robert Muir)

* SOLR-2986: Add MoreLikeThis to warning about features that require uniqueKey.

Also, change the warning to warn log level. (Shawn Heisey via Mark Miller)

* SOLR-4163: README improvements (Shawn Heisey via hossman)

* SOLR-4248: "ant eclipse" should declare .svn directories as derived.

(Shawn Heisey via Mark Miller)

* SOLR-3279: Upgrade Carrot2 to 3.6.2 (Stanisław Osiński)

* SOLR-4254: Harden the 'leader requests replica to recover' code path.

(Mark Miller, yonik)

* SOLR-4226: Extract fl parsing code out of ReturnFields constructor.

(Ryan Ernst via Robert Muir)

* SOLR-4208: ExtendedDismaxQParserPlugin has been refactored to make

subclassing easier. (Tomás Fernández Löbbe, hossman)

* SOLR-3735: Relocate the example mime-to-extension mapping, and

upgrade Velocity Engine to 1.7 (ehatcher)

* SOLR-4287: Removed "apache-" prefix from Solr distribution and artifact

filenames. (Ryan Ernst, Robert Muir, Steve Rowe)

* SOLR-4016: Deduplication does not work with atomic/partial updates so

disallow atomic update requests which change signature generating fields.
(Joel Nothman, yonik, shalin)

* SOLR-4308: Remove the problematic and now unnecessary log4j-over-slf4j.

(Mark Miller)

================== 4.0.0 ==================

Versions of Major Components

---------------------
Apache Tika 1.2
Carrot2 3.5.0
Velocity 1.6.4 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.3.6

Upgrading from Solr 4.0.0-BETA

----------------------
In order to better support distributed search mode, the TermVectorComponent's
response format has been changed so that if the schema defines a
uniqueKeyField, then that field value is used as the "key" for each document in
its response section, instead of the internal lucene doc id. Users w/o a
uniqueKeyField will continue to see the same response format. See SOLR-3229
for more details.

If you are using SolrCloud's distributed update request capabilities and a non
string type id field, you must re-index.

Upgrading from Solr 4.0.0-ALPHA

----------------------

Solr is now much more strict about requiring that the uniqueKeyField feature
(if used) must refer to a field which is not multiValued. If you upgrade from
an earlier version of Solr and see an error that your uniqueKeyField "can not
be configured to be multivalued" please add 'multiValued="false"' to the
<field /> declaration for your uniqueKeyField. See SOLR-3682 for more details.

In addition, please review the notes above about upgrading from 4.0.0-BETA

Upgrading from Solr 3.6

----------------------

* The Lucene index format has changed and as a result, once you upgrade,
previous versions of Solr will no longer be able to read your indices.
In a master/slave configuration, all searchers/slaves should be upgraded
before the master. If the master were to be updated first, the older
searchers would not be able to read the new index format.

* Setting abortOnConfigurationError=false is no longer supported

(since it has never worked properly). Solr will now warn you if
you attempt to set this configuration option at all. (see SOLR-1846)

* The default logic for the 'mm' param of the 'dismax' QParser has
been changed. If no 'mm' param is specified (either in the query,
or as a default in solrconfig.xml) then the effective value of the
'q.op' param (either in the query or as a default in solrconfig.xml
or from the 'defaultOperator' option in schema.xml) is used to
influence the behavior. If q.op is effectively "AND" then mm=100%.
If q.op is effectively "OR" then mm=0%. Users who wish to force the
legacy behavior should set a default value for the 'mm' param in
their solrconfig.xml file.

* The VelocityResponseWriter is no longer built into the core. Its JAR and
dependencies now need to be added (via <lib> or solr/home lib inclusion),
and it needs to be registered in solrconfig.xml like this:
<queryResponseWriter name="velocity" class="solr.VelocityResponseWriter"/>

* The update request parameter to choose Update Request Processor Chain is

renamed from "update.processor" to "update.chain". The old parameter was
deprecated but still working since Solr3.2, but is now removed
entirely.

* The <indexDefaults> and <mainIndex> sections of solrconfig.xml are discontinued

and replaced with the <indexConfig> section. There are also better defaults.
When migrating, if you don't know what your old settings mean, simply delete
both <indexDefaults> and <mainIndex> sections. If you have customizations,
 put them in <indexConfig> section - with same syntax as before.

* Two of the SolrServer subclasses in SolrJ were renamed/replaced.

CommonsHttpSolrServer is now HttpSolrServer, and
StreamingUpdateSolrServer is now ConcurrentUpdateSolrServer.

* The PingRequestHandler no longer looks for a <healthcheck/> option in the

(legacy) <admin> section of solrconfig.xml. Users who wish to take
advantage of this feature should configure a "healthcheckFile" init param
directly on the PingRequestHandler. As part of this change, relative file
paths have been fixed to be resolved against the data dir. See the example
solrconfig.xml and SOLR-1258 for more details.

* Due to low level changes to support SolrCloud, the uniqueKey field can no
longer be populated via <copyField/> or <field default=...> in the
schema.xml. Users wishing to have Solr automatically generate a uniqueKey
value when adding documents should instead use an instance of
solr.UUIDUpdateProcessorFactory in their update processor chain. See
SOLR-2796 for more details.

In addition, please review the notes above about upgrading from 4.0.0-BETA, and
4.0.0-ALPHA

Detailed Change List

----------------------

New Features
----------------------

* SOLR-3670: New CountFieldValuesUpdateProcessorFactory makes it easy to index

the number of values in another field for later use at query time. (hossman)

* SOLR-2768: new "mod(x,y)" function for computing the modulus of two value
sources. (hossman)

* SOLR-3238: Numerous small improvements to the Admin UI (steffkes)

* SOLR-3597: seems like a lot of wasted whitespace at the top of the admin screens
(steffkes)

* SOLR-3304: Added Solr adapters for Lucene 4's new spatial module. With
SpatialRecursivePrefixTreeFieldType ("location_rpt" in example schema), it is
possible to index a variable number of points per document (and sort on them),
index not just points but any Spatial4j supported shape such as polygons, and
to query on these shapes too. Polygons requires adding JTS to the classpath.
(David Smiley)

* SOLR-3825: Added optional capability to log what ids are in a response

(Scott Stults via gsingers)

* SOLR-3821: Added 'df' to the UI Query form (steffkes)

* SOLR-3822: Added hover titles to the edismax params on the UI Query form
(steffkes)

Optimizations
----------------------

* SOLR-3715: improve concurrency of the transaction log by removing

 synchronization around log record serialization. (yonik)

* SOLR-3807: Currently during recovery we pause for a number of seconds after

waiting for the leader to see a recovering state so that any previous updates
will have finished before our commit on the leader - we don't need this wait
for peersync. (Mark Miller)

* SOLR-3837: When a leader is elected and asks replicas to sync back to him and
that fails, we should ask those nodes to recovery asynchronously rather than
synchronously. (Mark Miller)

* SOLR-3709: Cache the url list created from the ClusterState in CloudSolrServer
on each request. (Mark Miller)

Bug Fixes
----------------------

* SOLR-3685: Solr Cloud sometimes skipped peersync attempt and replicated instead
due
to tlog flags not being cleared when no updates were buffered during a previous
replication. (Markus Jelsma, Mark Miller, yonik)

* SOLR-3229: Fixed TermVectorComponent to work with distributed search

(Hang Xie, hossman)

* SOLR-3725: Fixed package-local-src-tgz target to not bring in unnecessary jars

and binary contents. (Michael Dodsworth via rmuir)

* SOLR-3649: Fixed bug in JavabinLoader that caused deleteById(List<String> ids)

to not work in SolrJ (siren)

* SOLR-3730: Rollback is not implemented quite right and can cause corner case
fails in
SolrCloud tests. (rmuir, Mark Miller)

* SOLR-2981: Fixed StatsComponent to no longer return duplicated information

when requesting multiple stats.facet fields.
(Roman Kliewer via hossman)

* SOLR-3743: Fixed issues with atomic updates and optimistic concurrency in

conjunction with stored copyField targets by making real-time get never
return copyField targets. (yonik)

* SOLR-3746: Proper error reporting if updateLog is configured w/o necessary

"_version_" field in schema.xml (hossman)

* SOLR-3745: Proper error reporting if SolrCloud mode is used w/o

necessary "_version_" field in schema.xml (hossman)

* SOLR-3770: Overseer may lose updates to cluster state (siren)

* SOLR-3721: Fix bug that could theoretically allow multiple recoveries to run
briefly at the same time if the recovery thread join call was interrupted.
(Per Steffensen, Mark Miller)

* SOLR-3782: A leader going down while updates are coming in can cause shard
inconsistency. (Mark Miller)
* SOLR-3611: We do not show ZooKeeper data in the UI for a node that has children.
(Mark Miller)

* SOLR-3789: Fix bug in SnapPuller that caused "internal" compression to fail.

(siren)

* SOLR-3790: ConcurrentModificationException could be thrown when using hl.fl=*.

Fixed in r1231606. (yonik, koji)

* SOLR-3668: DataImport : Specifying Custom Parameters (steffkes)

* SOLR-3793: UnInvertedField faceting cached big terms in the filter

cache that ignored deletions, leading to duplicate documents in search
later when a filter of the same term was specified.
(Günter Hipler, hossman, yonik)

* SOLR-3679: Core Admin UI gives no feedback if "Add Core" fails (steffkes,

hossman)

* SOLR-3795: Fixed LukeRequestHandler response to correctly return field name

strings in copyDests and copySources arrays (hossman)

* SOLR-3699: Fixed some Directory leaks when there were errors during SolrCore
or SolrIndexWriter initialization. (hossman)

* SOLR-3518: Include final 'hits' in log information when aggregating a

distributed request (Markus Jelsma via hossman)

* SOLR-3628: SolrInputField and SolrInputDocument are now consistently backed

by Collections passed in to setValue/setField, and defensively copy values
from Collections passed to addValue/addField
(Tom Switzer via hossman)

* SOLR-3595: CurrencyField now generates an appropriate error on schema init

if it is configured as multiValued - this has never been properly supported,
but previously failed silently in odd ways. (hossman)

* SOLR-3823: Fix 'bq' parsing in edismax. Please note that this required
reverting the negative boost support added by SOLR-3278 (hossman)

* SOLR-3827: Fix shareSchema=true in solr.xml

(Tomás Fernández Löbbe via hossman)

* SOLR-3809: Fixed config file replication when subdirectories are used

(Emmanuel Espina via hossman)

* SOLR-3828: Fixed QueryElevationComponent so that using 'markExcludes' does

not modify the result set or ranking of 'excluded' documents relative to
not using elevation at all. (Alexey Serba via hossman)

* SOLR-3569: Fixed debug output on distributed requests when there are no

results found. (David Bowen via hossman)

* SOLR-3811: Query Form using wrong values for dismax, edismax (steffkes)

* SOLR-3779: DataImportHandler's LineEntityProcessor when used in conjunction

with FileListEntityProcessor would only process the first file.
(Ahmet Arslan via James Dyer)
* SOLR-3791: CachedSqlEntityProcessor would throw a NullPointerException when
a query returns a row with a NULL key. (Steffen Moelter via James Dyer)

* SOLR-3833: When a election is started because a leader went down, the new
leader candidate should decline if the last state they published was not
active. (yonik, Mark Miller)

* SOLR-3836: When doing peer sync, we should only count sync attempts that
cannot reach the given host as success when the candidate leader is
syncing with the replicas - not when replicas are syncing to the leader.
(Mark Miller)

* SOLR-3835: In our leader election algorithm, if on connection loss we found

we did not create our election node, we should retry, not throw an exception.
(Mark Miller)

* SOLR-3834: A new leader on cluster startup should also run the leader sync
process in case there was a bad cluster shutdown. (Mark Miller)

* SOLR-3772: On cluster startup, we should wait until we see all registered

replicas before running the leader process - or if they all do not come up,
N amount of time. (Mark Miller)

* SOLR-3756: If we are elected the leader of a shard, but we fail to publish

this for any reason, we should clean up and re trigger a leader election.
(Mark Miller)

* SOLR-3812: ConnectionLoss during recovery can cause lost updates, leading to

shard inconsistency. (Mark Miller)

* SOLR-3813: When a new leader syncs, we need to ask all shards to sync back,
not just those that are active. (Mark Miller)

* SOLR-3641: CoreContainer is not persisting roles core attribute.

(hossman, Mark Miller)

* SOLR-3527: SolrCmdDistributor drops some of the important commit attributes

(maxOptimizeSegments, softCommit, expungeDeletes) when sending a commit to
replicas. (Andy Laird, Tomás Fernández Löbbe, Mark Miller)

* SOLR-3844: SolrCore reload can fail because it tries to remove the index
write lock while already holding it. (Mark Miller)

* SOLR-3831: Atomic updates do not distribute correctly to other nodes.

(Jim Musil, Mark Miller)

* SOLR-3465: Replication causes two searcher warmups.

(Michael Garski, Mark Miller)

* SOLR-3645: /terms should default to distrib=false. (Nick Cotton, Mark Miller)

* SOLR-3759: Various fixes to the example-DIH configs (Ahmet Arslan, hossman)

* SOLR-3777: Dataimport-UI does not send unchecked checkboxes (Glenn MacStravic

via steffkes)

* SOLR-3850: DataImportHandler "cacheKey" parameter was incorrectly renamed

"cachePk"
(James Dyer)
* SOLR-3087: Fixed DOMUtil so that code doing attribute validation will
automatically ignore nodes in the reserved "xml" prefix - in particular this
fixes some bugs related to xinclude and fieldTypes.
(Amit Nithian, hossman)

* SOLR-3783: Fixed Pivot Faceting to work with facet.missing=true (hossman)

* SOLR-3869: A PeerSync attempt to its replicas by a candidate leader should

not fail on o.a.http.conn.ConnectTimeoutException. (Mark Miller)

* SOLR-3875: Fixed index boosts on multi-valued fields when docBoost is used

(hossman)

* SOLR-3878: Exception when using open-ended range query with CurrencyField

(janhoy)

* SOLR-3891: CacheValue in CachingDirectoryFactory cannot be used outside of

solr.core package. (phunt via Mark Miller)

* SOLR-3892: Inconsistent locking when accessing cache in CachingDirectoryFactory

from RAMDirectoryFactory and MockDirectoryFactory. (phunt via Mark Miller)

* SOLR-3883: Distributed indexing forwards non-applicable request params.

(Dan Sutton, Per Steffensen, yonik, Mark Miller)

* SOLR-3903: Fixed MissingFormatArgumentException in ConcurrentUpdateSolrServer

(hossman)

* SOLR-3916: Fixed whitespace bug in parsing the fl param (hossman)

Other Changes
----------------------

* SOLR-3690: Fixed binary release packages to include dependencies needed for

the solr-test-framework (hossman)

* SOLR-2857: The /update/json and /update/csv URLs were restored to aid

in the migration of existing clients. (yonik)

* SOLR-3691: SimplePostTool: Mode for crawling/posting web pages

See https://solr.apache.org/guide/solr/latest/indexing-guide/post-tool.html for
examples (janhoy)

* SOLR-3707: Upgrade Solr to Tika 1.2 (janhoy)

* SOLR-2747: Updated changes2html.pl to handle Solr's CHANGES.txt; added

target 'changes-to-html' to solr/build.xml.
(Steve Rowe, Robert Muir)

* SOLR-3752: When a leader goes down, have the Overseer clear the leader state
in cluster.json (Mark Miller)

* SOLR-3751: Add defensive checks for SolrCloud updates and requests that ensure
the local state matches what we can tell the request expected. (Mark Miller)

* SOLR-3773: Hash based on the external String id rather than the indexed
representation for distributed updates. (Michael Garski, yonik, Mark Miller)
* SOLR-3780: Maven build: Make solrj tests run separately from solr-core.
(Steve Rowe)

* SOLR-3772: Optionally, on cluster startup, we can wait until we see all

registered
replicas before running the leader process - or if they all do not come up,
N amount of time. (Jan Høydahl, Per Steffensen, Mark Miller)

* SOLR-3750: Optionally, on session expiration, we can explicitly wait some time

before
running the leader sync process so that we are sure every node participates.
(Per Steffensen, Mark Miller)

* SOLR-3824: Velocity: Error messages from search not displayed (janhoy)

* SOLR-3826: Test framework improvements for specifying coreName on initCore

(Amit Nithian, hossman)

* SOLR-3749: Allow default UpdateLog syncLevel to be configured by

solrconfig.xml (Raintung Li, Mark Miller)

* SOLR-3845: Rename numReplicas to replicationFactor in Collections API.

(yonik, Mark Miller)

* SOLR-3815: SolrCloud - Add properties such as "range" to shards, which changes

the clusterstate.json and puts the shard replicas under "replicas". (yonik)

* SOLR-3871: SyncStrategy should use an executor for the threads it creates to

request recoveries. (Mark Miller)

* SOLR-3870: SyncStrategy should have a close so it can abort earlier on

shutdown. (Mark Miller)

================== 4.0.0-BETA ===================

Versions of Major Components

---------------------
Apache Tika 1.1
Carrot2 3.5.0
Velocity 1.6.4 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.3.6

Upgrading from Solr 4.0.0-ALPHA

----------------------

Solr is now much more strict about requiring that the uniqueKeyField feature
(if used) must refer to a field which is not multiValued. If you upgrade from
an earlier version of Solr and see an error that your uniqueKeyField "can not
be configured to be multivalued" please add 'multiValued="false"' to the
<field /> declaration for your uniqueKeyField. See SOLR-3682 for more details.

Detailed Change List

----------------------

New Features
----------------------
* LUCENE-4201: Added JapaneseIterationMarkCharFilterFactory to normalize Japanese
iteration marks. (Robert Muir, Christian Moen)

* SOLR-1856: In Solr Cell, literals should override Tika-parsed values.

Patch adds a param "literalsOverride" which defaults to true, but can be set
to "false" to let Tika-parsed values be appended to literal values (Chris Harris,
janhoy)

* SOLR-3488: Added a Collection management API for SolrCloud.

(Tommaso Teofili, Sami Siren, yonik, Mark Miller)

* SOLR-3559: Full deleteByQuery support with SolrCloud distributed indexing.

All replicas of a shard will be consistent, even if updates arrive in a
different order on different replicas. (yonik)

* SOLR-1929: Index encrypted documents with ExtractingUpdateRequestHandler.

By supplying resource.password=<mypw> or specifying an external file with regular
expressions matching file names, Solr will decrypt and index PDFs and DOCX
formats.
(janhoy, Yiannis Pericleous)

* SOLR-3562: Add options to remove instance dir or data dir on core unload.
(Mark Miller, Per Steffensen)

* SOLR-2702: The default directory factory was changed to

NRTCachingDirectoryFactory
which wraps the StandardDirectoryFactory and caches small files for improved
Near Real-time (NRT) performance. (Mark Miller, yonik)

* SOLR-2616: Include a sample java util logging configuration file.

(David Smiley, Mark Miller)

* SOLR-3460: Add cloud-scripts directory and a zkcli.sh|bat tool for easy scripting
and interaction with ZooKeeper. (Mark Miller)

* SOLR-1725: StatelessScriptUpdateProcessorFactory allows users to implement

the full ScriptUpdateProcessor API using any scripting language with a
javax.script.ScriptEngineFactory
(Uri Boness, ehatcher, Simon Rosenthal, hossman)

* SOLR-139: Change to updateable documents to create the document if it doesn't

already exist. To assert that the document must exist, use the optimistic
concurrency feature by specifying a _version_ of 1. (yonik)

* LUCENE-2510, LUCENE-4044: Migrated Solr's Tokenizer-, TokenFilter-, and

CharFilterFactories to the lucene-analysis module. To add new analysis
modules to Solr (like ICU, SmartChinese, Morfologik,...), just drop in
the JAR files from Lucene's binary distribution into your Solr instance's
lib folder. The factories are automatically made available with SPI.
(Chris Male, Robert Muir, Uwe Schindler)

* SOLR-3634, SOLR-3635: CoreContainer and CoreAdminHandler will now remember

and report back information about failures to initialize SolrCores. These
failures will be accessible from the web UI and CoreAdminHandler STATUS
command until they are "reset" by creating/renaming a SolrCore with the
same name. (hossman, steffkes)

* SOLR-1280: Added commented-out example of the new script update processor

 to the example configuration. (ehatcher)

* SOLR-3672: SimplePostTool: Improvements for posting files

Support for auto mode, recursive and wildcards (janhoy)

Optimizations
----------------------

* SOLR-3708: Add hashCode to ClusterState so that structures built based on the

ClusterState can be easily cached. (Mark Miller)

* SOLR-3709: Cache the url list created from the ClusterState in CloudSolrServer on
each
request. (Mark Miller, yonik)

* SOLR-3710: Change CloudSolrServer so that update requests are only sent to

leaders by
default. (Mark Miller)

Bug Fixes
----------------------

* SOLR-3582: Our ZooKeeper watchers respond to session events as if they are change
events,
creating undesirable side effects. (Trym R. Møller, Mark Miller)

* SOLR-3467: ExtendedDismax escaping is missing several reserved characters

(Michael Dodsworth via janhoy)

* SOLR-3587: After reloading a SolrCore, the original Analyzer is still used rather
than a new
one. (Alexey Serba, yonik, rmuir, Mark Miller)

* LUCENE-4185: Fix a bug where CharFilters were wrongly being applied twice.
(Michael Froh, rmuir)

* SOLR-3610: After reloading a core, indexing would fail on any newly added fields
to the schema. (Brent Mills, rmuir)

* SOLR-3377: edismax fails to correctly parse a fielded query wrapped by parens.

This regression was introduced in 3.6. (Bernd Fehling, Jan Høydahl, yonik)

* SOLR-3621: Fix rare concurrency issue when opening a new IndexWriter for
replication or rollback.
(Mark Miller)

* SOLR-1781: Replication index directories not always cleaned up.

(Markus Jelsma, Terje Sten Bjerkseth, Mark Miller)

* SOLR-3639: Update ZooKeeper to 3.3.6 for a variety of bug fixes. (Mark Miller)

* SOLR-3629: Typo in solr.xml persistence when overriding the solrconfig.xml

file name using the "config" attribute prevented the override file from being
used. (Ryan Zezeski, hossman)

* SOLR-3642: Correct broken check for multivalued fields in stats.facet

(Yandong Yao, hossman)

* SOLR-3660: Velocity: Link to admin page broken (janhoy)

* SOLR-3658: Adding thousands of docs with one UpdateProcessorChain instance can
briefly create
spikes of threads in the thousands. (yonik, Mark Miller)

* SOLR-3656: A core reload now always uses the same dataDir. (Mark Miller, yonik)

* SOLR-3662: Core reload bugs: a reload always obtained a non-NRT searcher, which
could go back in time with respect to the previous core's NRT searcher.
Versioning
did not work correctly across a core reload, and update handler synchronization
was changed to synchronize on core state since more than on update handler
can coexist for a single index during a reload. (yonik)

* SOLR-3663: There are a couple of bugs in the sync process when a leader goes down
and a
new leader is elected. (Mark Miller)

* SOLR-3623: Fixed inconsistent treatment of third-party dependencies for

solr contribs analysis-extras & uima (hossman)

* SOLR-3652: Fixed range faceting to error instead of looping infinitely

when 'gap' is zero -- or effectively zero due to floating point arithmetic
underflow. (hossman)

* SOLR-3648: Fixed VelocityResponseWriter template loading in SolrCloud mode.

For the example configuration, this means /browse now works with SolrCloud.
(janhoy, ehatcher)

* SOLR-3677: Fixed misleading error message in web ui to distinguish between

no SolrCores loaded vs. no /admin/ handler available.
(hossman, steffkes)

* SOLR-3428: SolrCmdDistributor flushAdds/flushDeletes can cause repeated

adds/deletes to be sent (Mark Miller, Per Steffensen)

* SOLR-3647: DistributedQueue should use our Solr zk client rather than the std zk
client. ZooKeeper expiration can be permanent otherwise. (Mark Miller)

Other Changes
----------------------

* SOLR-3524: Make discarding punctuation configurable in JapaneseTokenizerFactory.

The default is to discard punctuation, but this is overridable as an expert
option.
(Kazuaki Hiraga, Jun Ohtani via Christian Moen)

* SOLR-1770: Move the default core instance directory into a collection1 folder.
(Mark Miller)

* SOLR-3355: Add shard and collection to SolrCore statistics. (Michael Garski, Mark
Miller)

* SOLR-3575: solr.xml should default to persist=true (Mark Miller)

* SOLR-3563: Unloading all cores in a SolrCloud collection will now cause the
removal of
that collection's meta data from ZooKeeper. (Mark Miller, Per Steffensen)
* SOLR-3599: Add zkClientTimeout to solr.xml so that it's obvious how to change it
and so
that you can change it with a system property. (Mark Miller)

* SOLR-3609: Change Solr's expanded webapp directory to be at a consistent path

called
solr-webapp rather than a temporary directory. (Mark Miller)

* SOLR-3600: Raise the default zkClientTimeout from 10 seconds to 15 seconds. (Mark

Miller)

* SOLR-3215: Clone SolrInputDocument when distrib indexing so that update

processors after
the distrib update process do not process the document twice. (Mark Miller)

* SOLR-3683: Improved error handling if an <analyzer> contains both an

explicit class attribute, as well as nested factories. (hossman)

* SOLR-3682: Fail to parse schema.xml if uniqueKeyField is multivalued (hossman)

* SOLR-2115: DIH no longer requires the "config" parameter to be specified in

solrconfig.xml.
Instead, the configuration is loaded and parsed with every import. This allows
the use of
a different configuration with each import, and makes correcting configuration
errors simpler.
Also, the configuration itself can be passed using the "dataConfig" parameter
rather than
using a file (this previously worked in debug mode only). When configuration
errors are
encountered, the error message is returned in XML format. (James Dyer)

* SOLR-3439: Make SolrCell easier to use out of the box. Also improves "/browse" to
display
rich-text documents correctly, along with facets for author and content_type.
With the new "content" field, highlighting of body is supported. See also SOLR-
3672 for
easier posting of a whole directory structure. (Jack Krupansky, janhoy)

* SOLR-3579: SolrCloud view should default to the graph view rather than tree view.
(steffkes, Mark Miller)

================== 4.0.0-ALPHA ==================

More information about this release, including any errata related to the
release notes, upgrade instructions, or other changes may be found online at:
https://cwiki.apache.org/confluence/display/solr/Solr4.0

Versions of Major Components

---------------------
Apache Tika 1.1
Carrot2 3.5.0
Velocity 1.6.4 and Velocity Tools 2.0
Apache UIMA 2.3.1
Apache ZooKeeper 3.3.4

Upgrading from Solr 3.6-dev

----------------------
* The Lucene index format has changed and as a result, once you upgrade,
previous versions of Solr will no longer be able to read your indices.
In a master/slave configuration, all searchers/slaves should be upgraded
before the master. If the master were to be updated first, the older
searchers would not be able to read the new index format.

* Setting abortOnConfigurationError=false is no longer supported

(since it has never worked properly). Solr will now warn you if
you attempt to set this configuration option at all. (see SOLR-1846)

* The default logic for the 'mm' param of the 'dismax' QParser has
been changed. If no 'mm' param is specified (either in the query,
or as a default in solrconfig.xml) then the effective value of the
'q.op' param (either in the query or as a default in solrconfig.xml
or from the 'defaultOperator' option in schema.xml) is used to
influence the behavior. If q.op is effectively "AND" then mm=100%.
If q.op is effectively "OR" then mm=0%. Users who wish to force the
legacy behavior should set a default value for the 'mm' param in
their solrconfig.xml file.

* The VelocityResponseWriter is no longer built into the core. Its JAR and
dependencies now need to be added (via <lib> or solr/home lib inclusion),
and it needs to be registered in solrconfig.xml like this:
<queryResponseWriter name="velocity" class="solr.VelocityResponseWriter"/>

* The update request parameter to choose Update Request Processor Chain is

renamed from "update.processor" to "update.chain". The old parameter was
deprecated but still working since Solr3.2, but is now removed
entirely.

* The <indexDefaults> and <mainIndex> sections of solrconfig.xml are discontinued

and replaced with the <indexConfig> section. There are also better defaults.
When migrating, if you don't know what your old settings mean, simply delete
both <indexDefaults> and <mainIndex> sections. If you have customizations,
put them in <indexConfig> section - with same syntax as before.

* Two of the SolrServer subclasses in SolrJ were renamed/replaced.

CommonsHttpSolrServer is now HttpSolrServer, and
StreamingUpdateSolrServer is now ConcurrentUpdateSolrServer.

* The PingRequestHandler no longer looks for a <healthcheck/> option in the

(legacy) <admin> section of solrconfig.xml. Users who wish to take
advantage of this feature should configure a "healthcheckFile" init param
directly on the PingRequestHandler. As part of this change, relative file
paths have been fixed to be resolved against the data dir. See the example
solrconfig.xml and SOLR-1258 for more details.

* Due to low level changes to support SolrCloud, the uniqueKey field can no
longer be populated via <copyField/> or <field default=...> in the
schema.xml. Users wishing to have Solr automatically generate a uniqueKey
value when adding documents should instead use an instance of
solr.UUIDUpdateProcessorFactory in their update processor chain. See
SOLR-2796 for more details.

Detailed Change List

----------------------
New Features
----------------------

* SOLR-3272: Solr filter factory for MorfologikFilter (Polish lemmatisation).

(Rafał Kuć via Dawid Weiss, Steven Rowe, Uwe Schindler).

* SOLR-571: The autowarmCount for LRUCaches (LRUCache and FastLRUCache) now

supports "percentages" which get evaluated relative the current size of
the cache when warming happens.
(Tomás Fernández Löbbe and hossman)

* SOLR-1932: New relevancy function queries: termfreq, tf, docfreq, idf

norm, maxdoc, numdocs. (yonik)

* SOLR-1665: Add debug component options for timings, results and query info only
(gsingers, hossman, yonik)

* SOLR-2112: Solrj API now supports streaming results. (ryan)

* SOLR-792: Adding PivotFacetComponent for Hierarchical faceting

(ehatcher, Jeremy Hinegardner, Thibaut Lassalle, ryan)

* LUCENE-2507, SOLR-2571, SOLR-2576: Added DirectSolrSpellChecker, which uses

Lucene's
DirectSpellChecker to retrieve correction candidates directly from the term
dictionary using
levenshtein automata. (James Dyer, rmuir)

* SOLR-1873, SOLR-2358: SolrCloud - added shared/central config and core/shard

management via zookeeper,
built-in load balancing, and distributed indexing.
(Jamie Johnson, Sami Siren, Ted Dunning, yonik, Mark Miller)
Additional Work:
- SOLR-2324: SolrCloud solr.xml parameters are not persisted by CoreContainer.
(Massimo Schiavon, Mark Miller)
- SOLR-2287: Allow users to query by multiple, compatible collections with
SolrCloud.
(Soheb Mahmood, Alex Cowell, Mark Miller)
- SOLR-2622: ShowFileRequestHandler does not work in SolrCloud mode.
(Stefan Matheis, Mark Miller)
- SOLR-3108: Error in SolrCloud's replica lookup code when replica's are hosted
in same Solr instance.
(Bruno Dumon, Sami Siren, Mark Miller)
- SOLR-3080: Remove shard info from zookeeper when SolrCore is explicitly
unloaded.
(yonik, Mark Miller, siren)
- SOLR-3437: Recovery issues a spurious commit to the cluster. (Trym R. Møller
via Mark Miller)
- SOLR-2822: Skip update processors already run on other nodes (hossman)

* SOLR-1566: Transforming documents in the ResponseWriters. This will allow

for more complex results in responses and open the door for function queries
as results.
(ryan with patches from grant, noble, cmale, yonik, Jan Høydahl,
Arul Kalaipandian, Luca Cavanna, hossman)
- SOLR-2037: Thanks to SOLR-1566, documents boosted by the
QueryElevationComponent
can be marked as boosted. (gsingers, ryan, yonik)
* SOLR-2396: Add CollationField, which is much more efficient than
the Solr 3.x CollationKeyFilterFactory, and also supports
Locale-sensitive range queries. (rmuir)

* SOLR-2338: Add support for using <similarity/> in a schema's fieldType,

for customizing scoring on a per-field basis. (hossman, yonik, rmuir)

* SOLR-2335: New 'field("...")' function syntax for referring to complex

field names (containing whitespace or special characters) in functions.

* SOLR-2383: /browse improvements: generalize range and date facet display

(Jan Høydahl via yonik)

* SOLR-2272: Pseudo-join queries / filters. Examples:

- To restrict to the set of parents with at least one blue-eyed child:
fq={!join from=parent to=name}eyes:blue
- To restrict to the set of children with at least one blue-eyed parent:
fq={!join from=name to=parent}eyes:blue
(yonik)

* SOLR-1942: Added the ability to select postings format per fieldType in

schema.xml
as well as support custom Codecs in solrconfig.xml.
(simonw via rmuir)

* SOLR-2136: Boolean type added to function queries, along with

new functions exists(), if(), and(), or(), xor(), not(), def(),
and true and false constants. (yonik)

* SOLR-2491: Add support for using spellcheck collation in conjunction

with grouping. Note that the number of hits returned for collations
is the number of ungrouped hits. (James Dyer via rmuir)

* SOLR-1298: Return FunctionQuery as pseudo field. The solr 'fl' param

now supports functions. For example: fl=id,sum(x,y) -- NOTE: only
functions with fast random access are recommended. (yonik, ryan)

* SOLR-705: Optionally return shard info with each document in distributed

search. Use fl=id,[shard] to return the shard url. (ryan)

* SOLR-2417: Add explain info directly to return documents using

?fl=id,[explain] (ryan)

* SOLR-2533: Converted ValueSource.ValueSourceSortField over to new rewriteable

Lucene
SortFields. ValueSourceSortField instances must be rewritten before they can be
used.
This is done by SolrIndexSearcher when necessary. (Chris Male).

* SOLR-2193, SOLR-2565: You may now specify a 'soft' commit when committing. This
will
use Lucene's NRT feature to avoid guaranteeing documents are on stable storage in
exchange
for faster reopen times. There is also a new 'soft' autocommit tracker that can
be
configured. (Mark Miller, Robert Muir)

* SOLR-2399: Updated Solr Admin interface. New look and feel with per core
administration
 and many new options. (Stefan Matheis via ryan)

* SOLR-1032: CSV handler now supports "literal.field_name=value" parameters.

(Simon Rosenthal, ehatcher)

* SOLR-2656: realtime-get, efficiently retrieves the latest stored fields for

specified
documents, even if they are not yet searchable (i.e. without reopening a
searcher)
(yonik)

* SOLR-2703: Added support for Lucene's "surround" query parser. (Simon Rosenthal,
ehatcher)

* SOLR-2754: Added factories for several ranking algorithms:

- BM25SimilarityFactory: Okapi BM25
- DFRSimilarityFactory: Divergence from Randomness models
- IBSimilarityFactory: Information-based models
- LMDirichletSimilarity: LM with Dirichlet smoothing
- LMJelinekMercerSimilarity: LM with Jelinek-Mercer smoothing
(David Mark Nemeskey, Robert Muir)

* SOLR-2134 Trie* fields should support sortMissingLast=true, and deprecate

Sortable* Field Types
(Ryan McKinley, Mike McCandless, Uwe Schindler, Erick Erickson)

* SOLR-2438 added MultiTermAwareComponent to the various classes to allow automatic

lowercasing
for multiterm queries (wildcards, regex, prefix, range, etc). You can now
optionally specify a
"multiterm" analyzer in our schema.xml, but Solr should "do the right thing" if
you don't
specify <analyzer type="multiterm"> (Pete Sturge Erick Erickson, Mentoring from
Seeley and Muir)

* SOLR-2481: Add support for commitWithin in DataImportHandler (Sami Siren via

yonik)

* SOLR-2992: Add support for IndexWriter.prepareCommit() via prepareCommit=true

on update URLs. (yonik)

* SOLR-2906: Added LFU cache options to Solr. (Shawn Heisey via Erick Erickson)

* SOLR-3069: Ability to add openSearcher=false to not open a searcher when doing

a hard commit. commitWithin now only invokes a softCommit. (yonik)

* SOLR-2802: New FieldMutatingUpdateProcessor and Factory to simplify the

development of UpdateProcessors that modify field values of documents as
they are indexed. Also includes several useful new implementations:
- RemoveBlankFieldUpdateProcessorFactory
- TrimFieldUpdateProcessorFactory
- HTMLStripFieldUpdateProcessorFactory
- RegexReplaceProcessorFactory
- FieldLengthUpdateProcessorFactory
- ConcatFieldUpdateProcessorFactory
- FirstFieldValueUpdateProcessorFactory
- LastFieldValueUpdateProcessorFactory
- MinFieldValueUpdateProcessorFactory
- MaxFieldValueUpdateProcessorFactory
 - TruncateFieldUpdateProcessorFactory
- IgnoreFieldUpdateProcessorFactory
(hossman, janhoy)

* SOLR-3120: Optional post filtering for spatial queries bbox and geofilt
for LatLonType. (yonik)

* SOLR-2459: Expose LogLevel selection with a RequestHandler rather then servlet

(Stefan Matheis, Upayavira, ryan)

* SOLR-3134: Include shard info in distributed response when shards.info=true

(Russell Black, ryan)

* SOLR-2898: Support grouped faceting. (Martijn van Groningen)

Additional Work:
- SOLR-3406: Extended grouped faceting support to facet.query and facet.range
parameters.
(David Boychuck, Martijn van Groningen)

* SOLR-2949: QueryElevationComponent is now supported with distributed search.

(Mark Miller, yonik)

* SOLR-3221: Added the ability to directly configure aspects of the concurrency

and thread-pooling used within distributed search in solr. This allows for finer
grained controlled and can be tuned by end users to target their own specific
requirements. This builds on the work of the HttpCommComponent and uses the same
configuration
block to configure the thread pool. The default configuration has
the same behaviour as solr 3.5, favouring throughput over latency. More
information can be found on the wiki (http://wiki.apache.org/solr/SolrConfigXml)
(Greg Bowyer)

* SOLR-3278: Negative boost support to the Extended Dismax Query Parser Boost Query
(bq).
(James Dyer)

* SOLR-3255: OpenExchangeRates.Org Exchange Rate Provider for CurrencyField

(janhoy)

* SOLR-3358: Logging events are captured and available from the /admin/logging
request handler. (ryan)

* SOLR-1535: PreAnalyzedField type provides a functionality to index (and

optionally store)
field content that was already processed and split into tokens using some
external processing
chain. Serialization format is pluggable, and defaults to JSON. (ab)

* SOLR-3363: Consolidated Exceptions in Analysis Factories so they only throw

InitializationExceptions (Chris Male)

* SOLR-2690: New support for a "TZ" request param which overrides the TimeZone
used when rounding Dates in DateMath expressions for the entire request
(all date range queries and date faceting is affected). The default TZ
is still UTC. (David Schlotfeldt, hossman)

* SOLR-3402: Analysis Factories are now configured with their Lucene Version
throw setLuceneMatchVersion, rather than through the Map passed to init.
Parsing and simple error checking for the Version is now done inside
 the code that creates the Analysis Factories. (Chris Male)

* SOLR-3178: Optimistic locking. If a _version_ is provided with an update

that does not match the version in the index, an HTTP 409 error (Conflict)
will result. (Per Steffensen, yonik)

* SOLR-139: Updateable documents. JSON Example:

{"id":"mydoc", "f1":{"set":10}, "f2":{"add":20}} will result in field "f1"
being set to 10, "f2" having an additional value of 20 added, and all
other existing fields unchanged. All source fields must be stored for
this feature to work correctly. (Ryan McKinley, Erik Hatcher, yonik)

* SOLR-2857: Support XML,CSV,JSON, and javabin in a single RequestHandler and

choose the correct ContentStreamLoader based on Content-Type header. This
also deprecates the existing [Xml,JSON,CSV,Binary,Xslt]UpdateRequestHandler.
(ryan)

* SOLR-2585: Context-Sensitive Spelling Suggestions & Collations. This adds

support
for the "spellcheck.alternativeTermCount" & "spellcheck.maxResultsForSuggest"
parameters, letting users receive suggestions even when all the queried terms
exist in the dictionary. This differs from "spellcheck.onlyMorePopular" in
that the suggestions need not consist entirely of terms with a greater document
frequency than the queried terms. (James Dyer)

* SOLR-2058: Edismax query parser to allow "phrase slop" to be specified per-field

on the pf/pf2/pf3 parameters using optional "FieldName~slop^boost" syntax. The
prior "FieldName^boost" syntax is still accepted. In such cases the value on the
"ps" parameter serves as the default slop. (Ron Mayer via James Dyer)

* SOLR-3495: New UpdateProcessors have been added to create default values for
configured fields. These works similarly to the <field default="..."/>
option in schema.xml, but are applied in the UpdateProcessorChain, so they
may be used prior to other UpdateProcessors, or to generate a uniqueKey field
value when using the DistributedUpdateProcessor (ie: SolrCloud)
TimestampUpdateProcessorFactory
UUIDUpdateProcessorFactory
DefaultValueUpdateProcessorFactory
(hossman)

* SOLR-2993: Add WordBreakSolrSpellChecker to offer suggestions by combining

adjacent
query terms and/or breaking terms into multiple words. This spellchecker can be
configured with a traditional checker (ie: DirectSolrSpellChecker). The results
are combined and collations can contain a mix of corrections from both
spellcheckers.
(James Dyer)

* SOLR-3508: Simplify JSON update format for deletes as well as allow

version specification for optimistic locking. Examples:
- {"delete":"myid"}
- {"delete":["id1","id2","id3"]}
- {"delete":{"id":"myid", "_version_":123456789}}
(yonik)

* SOLR-3211: Allow parameter overrides in conjunction with

"spellcheck.maxCollationTries".
To do so, use parameters starting with "spellcheck.collateParam." For instance,
to
 override the "mm" parameter, specify "spellcheck.collateParam.mm". This is
helpful
in cases where testing spellcheck collations for result counts should use
different
parameters from the main query (James Dyer)

* SOLR-2599: CloneFieldUpdateProcessorFactory provides similar functionality

to schema.xml's <copyField/> declaration but as an update processor that can
be combined with other processors in any order. (Jan Høydahl & hossman)

* SOLR-3351: eDismax: ps2 and ps3 params (janhoy)

* SOLR-3542: Add WeightedFragListBuilder for FVH and set it to default

fragListBuilder
in example solrconfig.xml. (Sebastian Lutze, koji)

* SOLR-2396: Add ICUCollationField to contrib/analysis-extras, which is much

more efficient than the Solr 3.x ICUCollationKeyFilterFactory, and also
supports Locale-sensitive range queries. (rmuir)

Optimizations
----------------------

* SOLR-1875: Per-segment field faceting for single valued string fields.

Enable with facet.method=fcs, control the number of threads used with
the "threads" local param on the facet.field param. This algorithm will
only be faster in the presence of rapid index changes. (yonik)

* SOLR-1904: When facet.enum.cache.minDf > 0 and the base doc set is a

SortedIntSet, convert to HashDocSet for better performance. (yonik)

* SOLR-2092: Speed up single-valued and multi-valued "fc" faceting. Typical

improvement is 5%, but can be much greater (up to 10x faster) when facet.offset
is very large (deep paging). (yonik)

* SOLR-2193, SOLR-2565: The default Solr update handler has been improved so
that it uses fewer locks, keeps the IndexWriter open rather than closing it
on each commit (ie commits no longer wait for background merges to complete),
works with SolrCore to provide faster 'soft' commits, and has an improved API
that requires less instanceof special casing. (Mark Miller, Robert Muir)
Additional Work:
- SOLR-2697: commit and autocommit operations don't reset
DirectUpdateHandler2.numDocsPending stats attribute.
(Alexey Serba, Mark Miller)

* SOLR-2950: The QueryElevationComponent now avoids using the FieldCache and

looking up
every document id (gsingers, yonik)

Bug Fixes
----------------------
* SOLR-3139: Make ConcurrentUpdateSolrServer send UpdateRequest.getParams()
as HTTP request params (siren)

* SOLR-3165: Cannot use DIH in Solrcloud + Zookeeper (Alexey Serba,

Mark Miller, siren)

* SOLR-3068: Occasional NPE in ThreadDumpHandler (siren)

* SOLR-2762: FSTLookup could return duplicate results or one results less
than requested. (David Smiley, Dawid Weiss)

* SOLR-2741: Bugs in facet range display in trunk (janhoy)

* SOLR-1908: Fixed SignatureUpdateProcessor to fail to initialize on

invalid config. Specifically: a signatureField that does not exist,
or overwriteDupes=true with a signatureField that is not indexed.
(hossman)

* SOLR-1824: IndexSchema will now fail to initialize if there is a

problem initializing one of the fields or field types. (hossman)

* SOLR-1928: TermsComponent didn't correctly break ties for non-text

fields sorted by count. (yonik)

* SOLR-2107: MoreLikeThisHandler doesn't work with alternate qparsers. (yonik)

* SOLR-2108: Fixed false positives when using wildcard queries on fields with
reversed
wildcard support. For example, a query of *zemog* would match documents that
contain
'gomez'. (Landon Kuhn via Robert Muir)

* SOLR-1962: SolrCore#initIndex should not use a mix of indexPath and newIndexPath

(Mark Miller)

* SOLR-2275: fix DisMax 'mm' parsing to be tolerant of whitespace

(Erick Erickson via hossman)

* SOLR-2193, SOLR-2565, SOLR-2651: SolrCores now properly share IndexWriters across

SolrCore reloads.
(Mark Miller, Robert Muir)
Additional Work:
- SOLR-2705: On reload, IndexWriterProvider holds onto the initial SolrCore it
was created with.
(Yury Kats, Mark Miller)

* SOLR-2682: Remove addException() in SimpleFacet. FacetComponent no longer catches

and embeds
exceptions occurred during facet processing, it throws HTTP 400 or 500 exceptions
instead. (koji)

* SOLR-2654: Directorys used by a SolrCore are now closed when they are no longer
used.
(Mark Miller)

* SOLR-2854: Now load URL content stream data (via stream.url) when called for
during request handling,
rather than loading URL content streams automatically regardless of use.
(David Smiley and Ryan McKinley via ehatcher)

* SOLR-2829: Fix problem with false-positives due to incorrect

equals methods. (Yonik Seeley, Hossman, Erick Erickson.
Marc Tinnemeyer caught the bug)

* SOLR-2848: Removed 'instanceof AbstractLuceneSpellChecker' hacks from distributed

spellchecking code,
 and added a merge() method to SolrSpellChecker instead. Previously if you
extended SolrSpellChecker
your spellchecker would not work in distributed fashion. (James Dyer via rmuir)

* SOLR-2509: StringIndexOutOfBoundsException in the spellchecker collate when the

term contains
a hyphen. (Thomas Gambier caught the bug, Steffen Godskesen did the patch, via
Erick Erickson)

* SOLR-1730: Made it clearer when a core failed to load as well as better logging
when the
QueryElevationComponent fails to properly initialize (gsingers)

* SOLR-1520: QueryElevationComponent now supports non-string ids (gsingers)

* SOLR-3037: When using binary format in solrj the codec screws up parameters
(Sami Siren, Jörg Maier via yonik)

* SOLR-3062: A join in the main query was not respecting any filters pushed
down to it via acceptDocs since LUCENE-1536. (Mike Hugo, yonik)

* SOLR-3214: If you use multiple fl entries rather than a comma separated list, all
but the first
entry can be ignored if you are using distributed search. (Tomás Fernández Löbbe
via Mark Miller)

* SOLR-3352: eDismax: pf2 should kick in for a query with 2 terms (janhoy)

* SOLR-3361: ReplicationHandler "maxNumberOfBackups" doesn't work if backups are

triggered on commit
(James Dyer, Tomás Fernández Löbbe)

* SOLR-2605: fixed tracking of the 'defaultCoreName' in CoreContainer so that

CoreAdminHandler could return consistent information regardless of whether
there is a a default core name or not. (steffkes, hossman)

* SOLR-3370: fixed CSVResponseWriter to respect globs in the 'fl' param

(Keith Fligg via hossman)

* SOLR-3436: Group count incorrect when not all shards are queried in the second
pass. (Francois Perron, Martijn van Groningen)

* SOLR-3454: Exception when using result grouping with main=true and using
wt=javabin. (Ludovic Boutros, Martijn van Groningen)

* SOLR-3446: Better errors when PatternTokenizerFactory is configured with

an invalid pattern, and include the 'name' whenever possible in plugin init
error messages. (hossman)

* LUCENE-4075: Cleaner path usage in TestXPathEntityProcessor

(Greg Bowyer via hossman)

* SOLR-2923: IllegalArgumentException when using useFilterForSortedQuery on an

empty index. (Adrien Grand via Mark Miller)

* SOLR-2352: Fixed TermVectorComponent so that it will not fail if the fl

param contains globs or pseudo-fields (hossman)

* SOLR-3541: add missing solrj dependencies to binary packages.

 (Thijs Vonk via siren)

* SOLR-3522: fixed parsing of the 'literal()' function (hossman)

* SOLR-3548: Fixed a bug in the cachability of queries using the {!join}

parser or the strdist() function, as well as some minor improvements to
the hashCode implementation of {!bbox} and {!geofilt} queries.
(hossman)

* SOLR-3470: contrib/clustering: custom Carrot2 tokenizer and stemmer factories

are respected now (Stanislaw Osinski, Dawid Weiss)

* SOLR-3430: Added a new DIH test against a real SQL database. Fixed problems
revealed by this new test related to the expanded cache support added to
3.6/SOLR-2382 (James Dyer)

* SOLR-1958: When using the MailEntityProcessor, import would fail if

fetchMailsSince was not specified. (Max Lynch via James Dyer)

* SOLR-4289: Admin UI - JVM memory bar - dark grey "used" width is too small
(steffkes, elyograg)

Other Changes
----------------------

* SOLR-1846: Eliminate support for the abortOnConfigurationError

option. It has never worked very well, and in recent versions of
Solr hasn't worked at all. (hossman)

* SOLR-1889: The default logic for the 'mm' param of DismaxQParser and
ExtendedDismaxQParser has been changed to be determined based on the
effective value of the 'q.op' param (hossman)

* SOLR-1946: Misc improvements to the SystemInfoHandler: /admin/system

(hossman)

* SOLR-2289: Tweak spatial coords for example docs so they are a bit
more spread out (Erick Erickson via hossman)

* SOLR-2288: Small tweaks to eliminate compiler warnings. primarily

using Generics where applicable in method/object declarations, and
adding @SuppressWarnings("unchecked") when appropriate (hossman)

* SOLR-2375: Suggester Lookup implementations now store trie data

and load it back on init. This means that large tries don't have to be
rebuilt on every commit or core reload. (ab)

* SOLR-2413: Support for returning multi-valued fields w/o <arr> tag

in the XMLResponseWriter was removed. XMLResponseWriter only
no longer work with values less then 2.2 (ryan)

* SOLR-2423: FieldType argument changed from String to Object

Conversion from SolrInputDocument > Object > Fieldable is now managed
by FieldType rather then DocumentBuilder. (ryan)

* SOLR-2461: QuerySenderListener and AbstractSolrEventListener are

now public (hossman)

* LUCENE-2995: Moved some spellchecker and suggest APIs to modules/suggest:

 HighFrequencyDictionary, SortedIterator, TermFreqIterator, and the
suggester APIs and implementations. (rmuir)

* SOLR-2576: Remove deprecated SpellingResult.add(Token, int).

(James Dyer via rmuir)

* LUCENE-3232: Moved MutableValue classes to new 'common' module. (Chris Male)

* LUCENE-2883: FunctionQuery, DocValues (and its impls), ValueSource (and its

impls) and BoostedQuery have been consolidated into the queries module. They
can now be found at o.a.l.queries.function.

* SOLR-2027: FacetField.getValues() now returns an empty list if there are no

values, instead of null (Chris Male)

* SOLR-1825: SolrQuery.addFacetQuery now enables facets automatically, like

addFacetField (Chris Male)

* SOLR-2663: FieldTypePluginLoader has been refactored out of IndexSchema

and made public. (hossman)

* SOLR-2331,SOLR-2691: Refactor CoreContainer's SolrXML serialization code and

improve testing
(Yury Kats, hossman, Mark Miller)

* SOLR-2698: Enhance CoreAdmin STATUS command to return index size.

(Yury Kats, hossman, Mark Miller)

* SOLR-2654: The same Directory instance is now always used across a SolrCore so
that
it's easier to add other DirectoryFactory's without static caching hacks.
(Mark Miller)

* LUCENE-3286: 'luke' ant target has been disabled due to incompatibilities with
XML
queryparser location (Chris Male)

* SOLR-1897: The data dir from the core descriptor should override the data dir
from
the solrconfig.xml rather than the other way round. (Mark Miller)

* SOLR-2756: Maven configuration: Excluded transitive stax:stax-api dependency

from org.codehaus.woodstox:wstx-asl dependency. (David Smiley via Steve Rowe)

* SOLR-2588: Moved VelocityResponseWriter back to contrib module in order to

remove it as a mandatory core dependency. (ehatcher)

* SOLR-2862: More explicit lexical resources location logged if Carrot2 clustering

extension is used. Fixed solr. impl. of IResource and IResourceLookup. (Dawid
Weiss)

* SOLR-1123: Changed JSONResponseWriter to now use application/json as its Content-

Type
by default. However the Content-Type can be overwritten and is set to text/plain
in
the example configuration. (Uri Boness, Chris Male)

* SOLR-2607: Removed deprecated client/ruby directory, which included solr-ruby and

flare.
 (ehatcher)

* SOLR-3032: logOnce from SolrException logOnce and all the supporting

structure is gone. abortOnConfigurationError is also gone as it is no longer
referenced.
Errors should be caught and logged at the top-most level or logged and NOT
propagated up the
chain. (Erick Erickson)

* SOLR-2105: Remove support for deprecated "update.processor" (since 3.2), in favor

of
"update.chain" (janhoy)

* SOLR-3005: Default QueryResponseWriters are now initialized via init() with an

empty
NamedList. (Gasol Wu, Chris Male)

* SOLR-2607: Removed obsolete client/ folder (ehatcher, Eric Pugh, janhoy)

* SOLR-3202, SOLR-3244: Dropping Support for JSP. New Admin UI is all client side
(ryan, Aliaksandr Zhuhrou, Uwe Schindler)

* SOLR-3159: Upgrade example and tests to run with Jetty 8 (ryan)

* SOLR-3254: Upgrade Solr to Tika 1.1 (janhoy)

* SOLR-3329: Dropped getSourceID() from SolrInfoMBean and using

getClass().getPackage().getSpecificationVersion() for Version. (ryan)

* SOLR-3302: Upgraded SLF4j to version 1.6.4 (hossman)

* SOLR-3322: Add more context to IndexReaderFactory.newReader (ab)

* SOLR-3343: Moved FastWriter, FileUtils, RegexFileFilter, RTimer and

SystemIdResolver
from org.apache.solr.common to org.apache.solr.util (Chris Male)

* SOLR-3357: ResourceLoader.newInstance now accepts a Class representation of the

expected
instance type (Chris Male)

* SOLR-3388: HTTP caching is now disabled by default for RequestUpdateHandlers.

(ryan)

* SOLR-3309: web.xml now specifies metadata-complete=true (which requires

Servlet 2.5) to prevent servlet containers from scanning class annotations
on startup. This allows for faster startup times on some servlet containers.
(Bill Bell, hossman)

* SOLR-1893: Refactored some common code from LRUCache and FastLRUCache into
SolrCacheBase (Tomás Fernández Löbbe via hossman)

* SOLR-3403: Deprecated Analysis Factories now log their own deprecation messages.
No logging support is provided by Factory parent classes. (Chris Male)

* SOLR-1258: PingRequestHandler is now directly configured with a

"healthcheckFile" instead of looking for the legacy
<admin><healthcheck/></admin> syntax. Filenames specified as relative
paths have been fixed so that they are resolved against the data dir
 instead of the CWD of the java process. (hossman)

* SOLR-3083: JMX beans now report Numbers as numeric values rather then String
(Tagged Siteops, Greg Bowyer via ryan)

* SOLR-2796: Due to low level changes to support SolrCloud, the uniqueKey

field can no longer be populated via <copyField/> or <field default=...>
in the schema.xml.

* SOLR-3534: The Dismax and eDismax query parsers will fall back on the 'df'
parameter
when 'qf' is absent. And if neither is present nor the schema default search
field
then an exception will be thrown now. (dsmiley)

* SOLR-3262: The "threads" feature of DIH is removed (deprecated in Solr 3.6)

(James Dyer)

* SOLR-3422: Refactored DIH internal data classes. All entities in

data-config.xml must have a name (James Dyer)

Documentation
----------------------

* SOLR-2232: Improved README info on solr.solr.home in examples

(Eric Pugh and hossman)

================== 3.6.2 ==================

Bug Fixes
----------------------
* SOLR-3790: ConcurrentModificationException could be thrown when using hl.fl=*.
(yonik, koji)

* SOLR-3589: Edismax parser does not honor mm parameter if analyzer splits a token.
(Tom Burton-West, Robert Muir)

================== 3.6.1 ==================

More information about this release, including any errata related to the
release notes, upgrade instructions, or other changes may be found online at:
https://wiki.apache.org/solr/Solr3.6.1

Bug Fixes

* LUCENE-3969: Throw IAE on bad arguments that could cause confusing errors in
PatternTokenizer. CommonGrams populates PositionLengthAttribute correctly.
(Uwe Schindler, Mike McCandless, Robert Muir)

* SOLR-3361: ReplicationHandler "maxNumberOfBackups" doesn't work if backups are

triggered on commit
(James Dyer, Tomás Fernández Löbbe)

* SOLR-3375: Fix charset problems with HttpSolrServer (Roger Håkansson, yonik,

siren)

* SOLR-3436: Group count incorrect when not all shards are queried in the second
pass. (Francois Perron, Martijn van Groningen)

* SOLR-3454: Exception when using result grouping with main=true and using
 wt=javabin. (Ludovic Boutros, Martijn van Groningen)

* SOLR-3489: Config file replication less error prone (Jochen Just via janhoy)

* SOLR-3477: SOLR does not start up when no cores are defined (Tomás Fernández
Löbbe via tommaso)

* SOLR-3470: contrib/clustering: custom Carrot2 tokenizer and stemmer factories

are respected now (Stanislaw Osinski, Dawid Weiss)

* SOLR-3360: More DIH bug fixes for the deprecated "threads" parameter.
(Mikhail Khludnev, Claudio R, via James Dyer)

* SOLR-3430: Added a new DIH test against a real SQL database. Fixed problems
revealed by this new test related to the expanded cache support added to
3.6/SOLR-2382 (James Dyer)

* SOLR-3336: SolrEntityProcessor substitutes most variables at query time.

(Michael Kroh, Lance Norskog, via Martijn van Groningen)

================== 3.6.0 ==================

More information about this release, including any errata related to the
release notes, upgrade instructions, or other changes may be found online at:
https://wiki.apache.org/solr/Solr3.6

Upgrading from Solr 3.5

----------------------
* SOLR-2983: As a consequence of moving the code which sets a MergePolicy from
SolrIndexWriter to SolrIndexConfig,
(custom) MergePolicies should now have an empty constructor; thus an IndexWriter
should not be passed as constructor
parameter but instead set using the setIndexWriter() method.

* As doGet() methods in SimplePostTool was changed to static, the client

applications of this
class need to be recompiled.

* In Solr version 3.5 and earlier, HTMLStripCharFilter had known bugs in the
character offsets it provided, triggering e.g. exceptions in highlighting.
HTMLStripCharFilter has been re-implemented, addressing this and other
issues. See the entry for LUCENE-3690 in the Bug Fixes section below for a
detailed list of changes. For people who depend on the behavior of
HTMLStripCharFilter in Solr version 3.5 and earlier: the old implementation
(bugs and all) is preserved as LegacyHTMLStripCharFilter.

* As of Solr 3.6, the <indexDefaults> and <mainIndex> sections of solrconfig.xml

are deprecated
and replaced with a new <indexConfig> section. Read more in SOLR-1052 below.

* SOLR-3040: The DIH's admin UI (dataimport.jsp) now requires DIH request handlers
to start with
a '/'. (dsmiley)

* SOLR-3161: <requestDispatcher handleSelect="false"> is now the default. An

existing config will
probably work as-is because handleSelect was explicitly enabled in default
configs. HandleSelect
makes /select work as well as enables the 'qt' parameter. Instead, consider
explicitly
configuring /select as is done in the example solrconfig.xml, and register your
other search
handlers with a leading '/' which is a recommended practice. (David Smiley, Erik
Hatcher)

* SOLR-3161: Don't use the 'qt' parameter with a leading '/'. It probably won't
work in 4.0
and it's now limited in 3.6 to SearchHandler subclasses that aren't lazy-loaded.

* SOLR-2724: Specifying <defaultSearchField> and <solrQueryParser

defaultOperator="..."/> in
schema.xml is now considered deprecated. Instead you are encouraged to specify
these via the "df"
and "q.op" parameters in your request handler definition. (David Smiley)

* Bugs found and fixed in the SignatureUpdateProcessor that previously caused

some documents to produce the same signature even when the configured fields
contained distinct (non-String) values. Users of SignatureUpdateProcessor
are strongly advised that they should re-index as document signatures may
have now changed. (see SOLR-3200 & SOLR-3226 for details)

New Features
----------------------
* SOLR-2020: Add Java client that uses Apache Http Components http client (4.x).
(Chantal Ackermann, Ryan McKinley, Yonik Seeley, siren)

* SOLR-2854: Now load URL content stream data (via stream.url) when called for
during request handling,
rather than loading URL content streams automatically regardless of use.
(David Smiley and Ryan McKinley via ehatcher)

* SOLR-2904: BinaryUpdateRequestHandler should be able to accept multiple update

requests from
a stream (shalin)

* SOLR-1565: StreamingUpdateSolrServer supports RequestWriter API and therefore,

javabin update
format (shalin)

* SOLR-2438 added MultiTermAwareComponent to the various classes to allow automatic

lowercasing
for multiterm queries (wildcards, regex, prefix, range, etc). You can now
optionally specify a
"multiterm" analyzer in our schema.xml, but Solr should "do the right thing" if
you don't
specify <fieldType="multiterm"> (Pete Sturge Erick Erickson, Mentoring from
Seeley and Muir)

* SOLR-2919: Added support for localized range queries when the analysis chain uses
CollationKeyFilter or ICUCollationKeyFilter. (Michael Sokolov, rmuir)

* SOLR-2982: Added BeiderMorseFilterFactory for Beider-Morse (BMPM) phonetic

encoder. Upgrades
commons-codec to version 1.6 (Brooke Schreier Ganz, rmuir)

* SOLR-1843: A new "rootName" attribute is now available when

configuring <jmx/> in solrconfig.xml. If this attribute is set,
Solr will use it as the root name for all MBeans Solr exposes via
 JMX. The default root name is "solr" followed by the core name.
(Constantijn Visinescu, hossman)

* SOLR-2906: Added LFU cache options to Solr. (Shawn Heisey via Erick Erickson)

* SOLR-3036: Ability to specify overwrite=false on the URL for XML updates.

(Sami Siren via yonik)

* SOLR-2603: Add the encoding function for alternate fields in highlighting.

(Massimo Schiavon, koji)

* SOLR-1729: Evaluation of NOW for date math is done only once per request for
consistency, and is also propagated to shards in distributed search.
Adding a parameter NOW=<time_in_ms> to the request will override the
current time. (Peter Sturge, yonik, Simon Willnauer)

* SOLR-1709: Distributed support for Date and Numeric Range Faceting

(Peter Sturge, David Smiley, hossman, Simon Willnauer)

* SOLR-3054, LUCENE-3671: Add TypeTokenFilterFactory that creates TypeTokenFilter

that filters tokens based on their TypeAttribute. (Tommaso Teofili via
Uwe Schindler)

* LUCENE-3305, SOLR-3056: Added Kuromoji morphological analyzer for Japanese.

See the 'text_ja' fieldtype in the example to get started.
(Christian Moen, Masaru Hasegawa via Robert Muir)

* SOLR-1860: StopFilterFactory, CommonGramsFilterFactory, and

CommonGramsQueryFilterFactory can optionally read stopwords in Snowball
format (specify format="snowball"). (Robert Muir)

* SOLR-3105: ElisionFilterFactory optionally allows the parameter

ignoreCase (default=false). (Robert Muir)

* LUCENE-3714: Add WFSTLookupFactory, a suggester that uses a weighted FST

for more fine-grained suggestions. (Mike McCandless, Dawid Weiss, Robert Muir)

* SOLR-3143: Add SuggestQueryConverter, a QueryConverter intended for

auto-suggesters. (Robert Muir)

* SOLR-3033: ReplicationHandler's backup command now supports a

'maxNumberOfBackups'
init param that can be used to delete all but the most recent N backups. (Torsten
Krah, James Dyer)

* SOLR-2202: Currency FieldType, whith support for currencies and exchange rates
(Greg Fodor & Andrew Morrison via janhoy, rmuir, Uwe Schindler)

* SOLR-3026: eDismax: Locking down which fields can be explicitly queried (user
fields aka uf)
(janhoy, hossmann, Tomás Fernández Löbbe)

* SOLR-2826: URLClassify Update Processor (janhoy)

* SOLR-2764: Create a NorwegianLightStemmer and NorwegianMinimalStemmer (janhoy)

* SOLR-3221: Added the ability to directly configure aspects of the concurrency

and thread-pooling used within distributed search in solr. This allows for finer
grained controlled and can be tuned by end users to target their own specific
 requirements. This builds on the work of the HttpCommComponent and uses the same
configuration
block to configure the thread pool. The default configuration has
the same behaviour as solr 3.5, favouring throughput over latency. More
information can be found on the wiki (http://wiki.apache.org/solr/SolrConfigXml)
(Greg Bowyer)

* SOLR-2001: The query component will substitute an empty query that matches
no documents if the query parser returns null. This also prevents an
exception from being thrown by the default parser if "q" is missing. (yonik)
- SOLR-435: if q is "" then it's also acceptable. (dsmiley, hoss)

* SOLR-2919: Added parametric tailoring options to ICUCollationKeyFilterFactory.

These can be used to customize range query/sort behavior, for example to
support numeric collation, ignore punctuation/whitespace, ignore accents but
not case, control whether upper/lowercase values are sorted first, etc. (rmuir)

* SOLR-2346: Add a chance to set content encoding explicitly via content type
of stream for extracting request handler. This is convenient when Tika's
auto detector cannot detect encoding, especially the text file is too short
to detect encoding. (koji)

* SOLR-1499: Added SolrEntityProcessor that imports data from another Solr core
or instance based on a specified query.
(Lance Norskog, Erik Hatcher, Pulkit Singhal, Ahmet Arslan, Luca Cavanna,
Martijn van Groningen)

* SOLR-3190: Minor improvements to SolrEntityProcessor. Add more consistency

between solr parameters and parameters used in SolrEntityProcessor and
ability to specify a custom HttpClient instance.
(Luca Cavanna via Martijn van Groningen)

* SOLR-2382: Added pluggable cache support to DIH so that any Entity can be
made cache-able by adding the "cacheImpl" parameter. Include
"SortedMapBackedCache" to provide in-memory caching (as previously this was
the only option when using CachedSqlEntityProcessor). Users can provide
their own implementations of DIHCache for other caching strategies.
Deprecate CachedSqlEntityProcessor in favor of specifing "cacheImpl" with
SqlEntityProcessor. Make SolrWriter implement DIHWriter and allow the
possibility of pluggable Writers (DIH writing to something other than Solr).
(James Dyer, Noble Paul)

Optimizations
----------------------
* SOLR-1931: Speedup for LukeRequestHandler and admin/schema browser. New parameter
reportDocCount defaults to 'false'. Old behavior still possible by specifying
this as 'true'
(Erick Erickson)

* SOLR-3012: Move System.getProperty("type") in postData() to main() and add type

argument so that
the client applications of SimplePostTool can set content type via method
argument. (koji)

* SOLR-2888: FSTSuggester refactoring: internal storage is now UTF-8,

external sorting (on disk) prevents OOMs even with large data sets
(the bottleneck is now FST construction), code cleanups and API cleanups.
(Dawid Weiss, Robert Muir)
Bug Fixes
----------------------
* SOLR-3187 SystemInfoHandler leaks filehandles (siren)

* LUCENE-3820: Fixed invalid position indexes by reimplementing

PatternReplaceCharFilter.
This change also drops real support for boundary characters -- all input is
prebuffered
for pattern matching. (Dawid Weiss)

* SOLR-3068: Fixed NPE in ThreadDumpHandler (siren)

* SOLR-2912: Fixed File descriptor leak in ShowFileRequestHandler (Michael Ryan,

shalin)

* SOLR-2819: Improved speed of parsing hex entities in HTMLStripCharFilter

(Bernhard Berger, hossman)

* SOLR-2509: StringIndexOutOfBoundsException in the spellchecker collate when the

term contains
a hyphen. (Thomas Gambier caught the bug, Steffen Godskesen did the patch, via
Erick Erickson)

* SOLR-2955: Fixed IllegalStateException when querying with group.sort=score desc

in sharded
environment. (Steffen Elberg Godskesen, Martijn van Groningen)

* SOLR-2956: Fixed inconsistencies in the flags (and flag key) reported by

the LukeRequestHandler (hossman)

* SOLR-1730: Made it clearer when a core failed to load as well as better logging
when the
QueryElevationComponent fails to properly initialize (gsingers)

* SOLR-1520: QueryElevationComponent now supports non-string ids (gsingers)

* SOLR-3024: Fixed JSONTestUtil.matchObj, in previous releases it was not

respecting the 'delta' arg (David Smiley via hossman)

* SOLR-2542: Fixed DIH Context variables which were broken for all scopes other
then SCOPE_ENTITY (Linbin Chen & Frank Wesemann via hossman)

* SOLR-3042: Fixed Maven Jetty plugin configuration.

(David Smiley via Steve Rowe)

* SOLR-2970: CSV ResponseWriter returns fields defined as stored=false in schema

(janhoy)

* LUCENE-3690, LUCENE-2208, SOLR-882, SOLR-42: Re-implemented

HTMLStripCharFilter as a JFlex-generated scanner and moved it to
lucene/contrib/analyzers/common/. See below for a list of bug fixes and
other changes. To get the same behavior as HTMLStripCharFilter in Solr
version 3.5 and earlier (including the bugs), use LegacyHTMLStripCharFilter,
which is the previous implementation.

Behavior changes from the previous version:

- Known offset bugs are fixed.

 - The "Mark invalid" exceptions reported in SOLR-1283 are no longer
triggered (the bug is still present in LegacyHTMLStripCharFilter).
- The character entity "&apos;" is now always properly decoded.
- More cases of <script> tags are now properly stripped.
- CDATA sections are now handled properly.
- Valid tag name characters now include the supplementary Unicode characters
from Unicode character classes [:ID_Start:] and [:ID_Continue:].
- Uppercase character entities "&QUOT;", "&COPY;", "&GT;", "&LT;", "&REG;",
and "&AMP;" are now recognized and handled as if they were in lowercase.
- The REPLACEMENT CHARACTER U+FFFD is now used to replace numeric character
entities for unpaired UTF-16 low and high surrogates (in the range
[U+D800-U+DFFF]).
- Properly paired numeric character entities for UTF-16 surrogates are now
converted to the corresponding code units.
- Opening tags with unbalanced quotation marks are now properly stripped.
- Literal "<" and ">" characters in opening tags, regardless of whether they
appear inside quotation marks, now inhibit recognition (and stripping) of
the tags. The only exception to this is for values of event-handler
attributes, e.g. "onClick", "onLoad", "onSelect".
- A newline '\n' is substituted instead of a space for stripped HTML markup.
- Nothing is substituted for opening and closing inline tags - they are
simply removed. The list of inline tags is (case insensitively): <a>,
<abbr>, <acronym>, <b>, <basefont>, <bdo>, <big>, <cite>, <code>, <dfn>,
<em>, <font>, <i>, <img>, <input>, <kbd>, <label>, <q>, <s>, <samp>,
<select>, <small>, <span>, <strike>, <strong>, <sub>, <sup>, <textarea>,
<tt>, <u>, and <var>.
- HTMLStripCharFilterFactory now handles HTMLStripCharFilter's "escapedTags"
feature: opening and closing tags with the given names, including any
attributes and their values, are left intact in the output.
(Steve Rowe)

* LUCENE-3717: Fixed offset bugs in TrimFilter, WordDelimiterFilter, and

HyphenatedWordsFilter where they would create invalid offsets in
some situations, leading to problems in highlighting. (Robert Muir)

* SOLR-2280: commitWithin ignored for a delete query (Juan Grande via janhoy)

* SOLR-3073: Fixed 'Invalid UUID string' error when having an UUID field as
the unique key and executing a distributed grouping request. (Devon Krisman,
Martijn van Groningen)

* SOLR-3084: Fixed initialization error when using

<queryResponseWriter default="true" ... /> (Bernd Fehling and hossman)

* SOLR-3109: Fixed numerous redundant shard requests when using distributed

grouping.
(rblack via Martijn van Groningen)

* SOLR-3052: Fixed typo in distributed grouping parameters.

(Martijn van Groningen, Grant Ingersoll)

* SOLR-2909: Add support for ResourceLoaderAware tokenizerFactories in synonym

filter factories. (Tom Klonikowski, Jun Ohtani via Koji Sekiguchi)

* SOLR-3168: ReplicationHandler "numberToKeep" & "maxNumberOfBackups" parameters

would keep only 1 backup, even if more than 1 was specified (Neil Hooey, James
Dyer)

* SOLR-3009: hitGrouped.vm isn't shipped with 3.x (ehatcher, janhoy)

* SOLR-3195: timeAllowed is ignored for grouping queries
(Russell Black via Martijn van Groningen)

* SOLR-2124: Do not log stack traces for "Service Disabled" / 503 Exceptions
(PingRequestHandler, etc)
(James Dyer, others)

* SOLR-3260: DataImportHandler: ScriptTransformer gives better error messages when

problems arise on initialization (no Script Engine, invalid script, etc). (James
Dyer)

* SOLR-2959: edismax now respects the magic fields '_val_' and '_query_'
(Michael Watts, hossman)

* SOLR-3074: fix SolrPluginUtils.docListToSolrDocumentList to respect the

list of fields specified. This fix also deprecates
DocumentBuilder.loadStoredFields which is not used anywhere in Solr,
and was fundamentally broken/bizarre.
(hossman, Ahmet Arslan)

* SOLR-2291: fix JSONWriter to respect field list when writing SolrDocuments

(Ahmet Arslan via hossman)

* SOLR-3264: Fix CoreContainer and SolrResourceLoader logging to be more

clear about when SolrCores are being created, and stop misleading people
about SolrCore instanceDir's being the "Solr Home Dir" (hossman)

* SOLR-3046: Fix whitespace typo in DIH response "Time taken" (hossman)

* SOLR-3261: Fix edismax to respect query operators when literal colons

are used in query string. (Juan Grande via hossman)

* SOLR-3226: Fix SignatureUpdateProcessor to no longer ignore non-String

field values (Spyros Kapnissis, hossman)

* SOLR-3200: Fix SignatureUpdateProcessor "all fields" mode to use all

fields of each document instead of the fields specified by the first
document indexed (Spyros Kapnissis via hossman)

* SOLR-3316: Distributed grouping failed when rows parameter was set to 0 and
sometimes returned a wrong hit count as matches. (Cody Young, Martijn van
Groningen)

* SOLR-3107: contrib/langid: When using the LangDetect implementation of

langid, set the random seed to 0, so that the same document is detected as
the same language with the same probability every time.
(Christian Moen via rmuir)

* SOLR-2937: Configuring the number of contextual snippets used for

search results clustering. The hl.snippets parameter is now respected
by the clustering plugin, can be overridden by carrot.summarySnippets
if needed (Stanislaw Osinski).

* SOLR-2938: Clustering on multiple fields. The carrot.title and

carrot.snippet can now take comma- or space-separated lists of
field names to cluster (Stanislaw Osinski).

* SOLR-2939: Clustering of multilingual search results. The document's

 language field be passed in the carrot.lang parameter, the carrot.lcmap
parameter enables mapping of language codes to ISO 639 (Stanislaw Osinski).

* SOLR-2940: Passing values for custom Carrot2 fields to Clustering component.

The custom field mapping are defined using the carrot.custom parameter
(Stanislaw Osinski).

* SOLR-2941: NullPointerException on clustering component initialization

when schema does not have a unique key field (Stanislaw Osinski).

* SOLR-2942: ClassCastException when passing non-textual fields to

clustering component (Stanislaw Osinski).

Other Changes
----------------------
* SOLR-2922: Upgrade commons-io and commons-lang to 2.1 and 2.6, respectively.
(koji)

* SOLR-2920: Refactor frequent conditional use of DefaultSolrParams and

AppendedSolrParams into factory methods.
(David Smiley via hossman)

* SOLR-3032: Deprecate logOnce from SolrException logOnce and all the supporting
structure will disappear in 4.0. Errors should be caught and logged at the
top-most level or logged and NOT propagated up the chain. (Erick Erickson)

* SOLR-2718: Add ability to lazy load response writers, defined with

startup="lazy".
(ehatcher)

* SOLR-2901: Upgrade Solr to Tika 1.0 (janhoy)

* SOLR-3059: Example XSL stylesheet for indexing query result XML (janhoy)

* SOLR-3097, SOLR-3105: Add analysis configurations for different languages to

the example. (Christian Moen, Robert Muir)

* SOLR-3005: Default QueryResponseWriters are now initialized via init() with an

empty
NamedList. (Gasol Wu, Chris Male)

* SOLR-3140: Upgrade schema version to 1.5, where omitNorms defaults to "true" for
all
primitive (non-analyzed) field types such as int, float, date, bool, string..
(janhoy)

* SOLR-3077: Better error messages when attempting to use "blank" field names
(Antony Stubbs via hossman)

* SOLR-2712: expecting fl=score to return all fields is now deprecated.

In solr 4.0, this will only return the score. (ryan)

* SOLR-3156: Check for Lucene directory locks at startup. In previous versions

this check was only performed during modifying (e.g. adding and deleting
documents) the index. (Luca Cavanna via Martijn van Groningen)

* SOLR-1052: Deprecated <indexDefaults> and <mainIndex> in solrconfig.xml

From now, all settings go in the new <indexConfig> tag, and some defaults are
 changed: useCompoundFile=false, ramBufferSizeMB=32, lockType=native, so that
the effect of NOT specifying <indexConfig> at all gives same result as the
example config used to give in 3.5 (janhoy, gsingers)

* SOLR-3294: In contrib/clustering/lib/, replaced the manually retrowoven

Java 1.5-compatible carrot2-core-3.5.0.jar (which is not publicly available,
except from the Solr Subversion repository), with newly released Java
1.5-compatible carrot2-core-3.5.0.1.jar (hosted on the Maven Central
repository). Also updated dependencies jackson-core-asl and
jackson-mapper-asl (both v1.5.2 -> v1.7.4). (Dawid Weiss, Steve Rowe)

* SOLR-3295: netcdf jar is excluded from the binary release (and disabled in
ivy.xml) because it requires java 6. If you want to parse this content with
extracting request handler and are willing to use java 6, just add the jar.
(rmuir)

* SOLR-3142: DIH Imports no longer default optimize to true, instead false.

If you want to force all segments to be merged into one, you can specify
this parameter yourself. NOTE: this can be very expensive operation and
usually does not make sense for delta-imports. (Robert Muir)

Build
----------------------
* SOLR-2487: Add build target to package war without slf4j jars (janhoy)

* SOLR-3112: Fix tests not to write to src/test-files (Luca Cavanna via Robert
Muir)

* LUCENE-3753: Restructure the Solr build system. (Steve Rowe)

* SOLR-3204: The packaged pre-release artifact of Commons CSV used the original
package name (org.apache.commons.csv). This created a compatibility issue as
the Apache Commons team works toward an official release of Commons CSV.
The source of Commons CSV was added under a separate package name to the
Solr source code. (Uwe Schindler, Chris Male, Emmanuel Bourg)

* LUCENE-3930: Changed build system to use Apache Ivy for retrival of 3rd
party JAR files. Please review README.txt for instructions.
(Robert Muir, Chris Male, Uwe Schindler, Steven Rowe, Hossman)

================== 3.5.0 ==================

New Features
----------------------
* SOLR-2749: Add boundary scanners for FastVectorHighlighter. <boundaryScanner/>
can be specified with a name in solrconfig.xml, and use hl.boundaryScanner=name
parameter to specify the named <boundaryScanner/>. (koji)

* SOLR-2066,SOLR-2776: Added support for distributed grouping.

(Martijn van Groningen, Jasper van Veghel, Matt Beaumont)

* SOLR-2769: Added factory for the new Hunspell stemmer capable of doing stemming
for 99 languages (janhoy, cmale)

* SOLR-1979: New contrib "langid". Adds language identification capabilities as an

Update Processor, using Tika's LanguageIdentifier or Cybozu language-detection
library (janhoy, Tommaso Teofili, gsingers)

* SOLR-2818: Added before/after count response parsing support for range facets in
 SolrJ. (Bernhard Frauendienst via Martijn van Groningen)

* SOLR-2276: Add support for cologne phonetic to PhoneticFilterFactory.

(Marc Pompl via rmuir)

* SOLR-1926: Add hl.q parameter. (koji)

* SOLR-2881: Numeric types now support sortMissingFirst/Last. This includes Trie

and date types
(Ryan McKinley, Mike McCandless, Uwe Schindler, Erick Erickson)

* SOLR-1023: StatsComponent now supports date fields and string fields.

(Chris Male, Mark Holland, Gunnlaugur Thor Briem, Ryan McKinley)

* SOLR-2578: ReplicationHandler's backup command now supports a 'numberToKeep'

request param that can be used to delete all but the most recent N backups.
(James Dyer via hossman)

* SOLR-2839: Add alternative implementation to contrib/langid supporting 53

languages, based on http://code.google.com/p/language-detection/ (rmuir)

Optimizations
----------------------

* SOLR-2742: SolrJ: Provide commitWithinMs as optional parameter for all add()

methods,
making the feature more conveniently accessible for developers (janhoy)

Bug Fixes
----------------------
* SOLR-2748: The CommitTracker used for commitWith or autoCommit by maxTime
could commit too frequently and could block adds until a new searcher was
registered. (yonik)

* SOLR-2726: Fixed NullPointerException when using spellcheck.q with Suggester.

(Bernd Fehling, valentin via rmuir)

* SOLR-2772: Fixed Date parsing/formatting of years 0001-1000 (hossman)

* SOLR-2763: Extracting update request handler throws exception and returns 400
when zero-length file posted using multipart form post (janhoy)

* SOLR-2780: Fixed issue where multi select facets didn't respect group.truncate
parameter.
(Martijn van Groningen, Ramzi Alqrainy)

* SOLR-2793: In rare cases (most likely during shutdown), a SolrIndexSearcher can

be left
open if the executor rejects a task. (Mark Miller)

* SOLR-2791: Replication: abortfetch command is broken if replication was started

by fetchindex command instead of a regular poll (Yury Kats via shalin)

* SOLR-2861: Fix extremely rare race condition on commit that can result
in a NPE (yonik)

* SOLR-2813: Fix HTTP error codes returned when requests contain strings that
can not be parsed as numbers for Trie fields. (Jeff Crump and hossman)
* SOLR-2902: List of collations are wrong parsed in SpellCheckResponse causing
a wrong number of collation results in the response.
(Bastiaan Verhoef, James Dyer via Simon Willnauer)

* SOLR-2875: Fix the incorrect url in DIH example tika-data-config.xml

(Shinichiro Abe via koji)

Other Changes
----------------------

* SOLR-2750: Make both "update.chain" and the deprecated "update.param" work

consistently everywhere; see also SOLR-2105. (Mark Miller, janhoy)

* LUCENE-3410: Deprecated the WordDelimiterFilter constructors accepting multiple

ints masquerading as booleans. Preferred constructor now accepts a single int
bitfield (Chris Male)

* SOLR-2758: Moved ConcurrentLRUCache from o.a.s.common.util package in the solrj

module to the o.a.s.util package in the Solr core module.
(David Smiley via Steve Rowe)

* SOLR-2766: Package individual javadoc sites for solrj and test-framework.

(Steve Rowe, Mike McCandless)

* SOLR-2771: Solr modules' tests should not depend on solr-core test classes;
move BufferingRequestProcessor from solr-core tests to test-framework so that
the Solr Cell module can use it. (janhoy, Steve Rowe)

* LUCENE-3457: Upgrade commons-compress to 1.2 (Doron Cohen)

* SOLR-2757: min() and max() functions now support an arbitrary number of

ValueSources (Bill Bell via hossman)

* SOLR-2372: Upgrade Solr to Tika 0.10 (janhoy)

* SOLR-2792: Allow case insensitive Hunspell stemming (janhoy, rmuir)

* SOLR-2862: More explicit lexical resources location logged if Carrot2 clustering

extension is used. Fixed solr. impl. of IResource and IResourceLookup. (Dawid
Weiss)

* SOLR-2849: Fix dependencies in Maven POMs. (David Smiley via Steve Rowe)

* SOLR-2591: Remove commitLockTimeout option from solrconfig.xml (Luca Cavanna via

Martijn van Groningen)

* SOLR-2746: Upgraded UIMA dependencies from *-2.3.1-SNAPSHOT.jar to *-2.3.1.jar.

================== 3.4.0 ==================

Upgrading from Solr 3.3

----------------------

* The Lucene index format has changed and as a result, once you upgrade,
previous versions of Solr will no longer be able to read your indices.
In a master/slave configuration, all searchers/slaves should be upgraded
before the master. If the master were to be updated first, the older
searchers would not be able to read the new index format.
* Previous versions of Solr silently allow and ignore some contradictory
properties specified in schema.xml. For example:
- indexed="false" omitNorms="false"
- indexed="false" omitTermFreqAndPositions="false"
Field property validation has now been fixed, to ensure that
contradictions like these now generate error messages. If users
have existing schemas that generate one of these new "conflicting
'false' field options for non-indexed field" error messages the
conflicting "omit*" properties can safely be removed, or changed to
"true" for consistent behavior with previous Solr versions. This
situation has now been fixed to cause an error on startup when these
contradictory options. See SOLR-2669.

* FacetComponent no longer catches and embeds exceptions occurred during facet

processing, it throws HTTP 400 or 500 exceptions instead.

New Features
----------------------

* SOLR-2540: CommitWithin as an Update Request parameter

You can now specify &commitWithin=N (ms) on the update request (janhoy)

* SOLR-2458: post.jar enhanced to handle JSON, CSV and <optimize> (janhoy)

* LUCENE-3234: add a new parameter hl.phraseLimit for FastVectorHighlighter speed

up.
(Mike Sokolov via koji)

* SOLR-2429: Ability to add cache=false to queries and query filters to avoid

using the filterCache or queryCache. A cost may also be specified and is used
to order the evaluation of non-cached filters from least to greatest cost .
For very expensive query filters (cost >= 100) if the query implements
the PostFilter interface, it will be used to obtain a Collector that is
checked only for documents that match the main query and all other filters.
The "frange" query now implements the PostFilter interface. (yonik)

* SOLR-2630: Added new XsltUpdateRequestHandler that works like

XmlUpdateRequestHandler but allows to transform the POSTed XML document
using XSLT. This allows to POST arbitrary XML documents to the update
handler, as long as you also provide a XSL to transform them to a valid
Solr input document. (Upayavira, Uwe Schindler)

* SOLR-2615: Log individual updates (adds and deletes) at the FINE level
before adding to the index. Fix a null pointer exception in logging
when there was no unique key. (David Smiley via yonik)

* LUCENE-2048: Added omitPositions to the schema, so you can omit position

information while still indexing term frequencies. (rmuir)

* SOLR-2584: add UniqFieldsUpdateProcessor that removes duplicate values in the

specified fields. (Elmer Garduno, koji)

* SOLR-2670: Added NIOFSDirectoryFactory (yonik)

* SOLR-2523: Added support in SolrJ to easily interact with range facets.

The range facet response can be parsed and is retrievable from the
QueryResponse class. The SolrQuery class has convenient methods for using
range facets. (Martijn van Groningen)
* SOLR-2637: Added support for group result parsing in SolrJ.
(Tao Cheng, Martijn van Groningen)

* SOLR-2665: Added post group faceting. Facet counts are based on the most
relevant document of each group matching the query. This feature has the
same impact on the StatsComponent. (Martijn van Groningen)

* SOLR-2675: CoreAdminHandler now allows arbitrary properties to be

specified when CREATEing a new SolrCore using property.* request
params. (Yury Kats, hossman)

* SOLR-2714: JSON update format - "null" field values are now dropped
instead of causing an exception. (Trygve Laugstøl, yonik)

Optimizations
----------------------

* LUCENE-3233: Improved memory usage, build time, and performance of

SynonymFilterFactory. (Mike McCandless, Robert Muir)

Bug Fixes
----------------------

* SOLR-2625: TermVectorComponent throws NPE if TF-IDF option is used without DF

option. (Daniel Erenrich, Simon Willnauer)

* SOLR-2631: PingRequestHandler should not allow to ping itself using "qt"

param to prevent infinite loop. (Edoardo Tosca, Uwe Schindler)

* SOLR-2636: Fix explain functionality for negative queries. (Tom Hill via yonik)

* SOLR-2538: Range Faceting on long/double fields could overflow if values

bigger then the max int/float were used.
(Erbi Hanka, hossman)

* SOLR-2230: CommonsHttpSolrServer.addFile could not be used to send

multiple files in a single request.
(Stephan Günther, hossman)

* SOLR-2541: PluginInfos was not correctly parsing <long/> tags when

initializing plugins
(Frank Wesemann, hossman)

* SOLR-2623: Solr JMX MBeans do not survive core reloads (Alexey Serba, shalin)

* Fixed grouping bug when start is bigger than rows and format is simple that zero
documents are returned even
if there are documents to display. (Martijn van Groningen, Nikhil Chhaochharia)

* SOLR-2564: Fixed ArrayIndexOutOfBoundsException when using simple format and

start > 0 (Martijn van Groningen, Matteo Melli)

* SOLR-2642: Fixed sorting by function when using grouping. (Thomas Heigl, Martijn
van Groningen)

* SOLR-2535: REGRESSION: in Solr 3.x and trunk the admin/file handler

fails to show directory listings (David Smiley, Peter Wolanin via Erick Erickson)
* SOLR-2545: ExternalFileField file parsing would fail if any key
contained an "=" character. It now only looks for the last "=" delimiter
prior to the float value.
(Markus Jelsma, hossman)

* SOLR-2662: When Solr is configured to have no queryResultCache, the

"start" parameter was not honored and the documents returned were
0 through start+offset. (Markus Jelsma, yonik)

* SOLR-2669: Fix backwards validation of field properties in

SchemaField.calcProps (hossman)

* SOLR-2676: Add "welcome-file-list" to solr.war so admin UI works correctly

in servlet containers such as WebSphere that do not use a default list
(Jay R. Jaeger, hossman)

* SOLR-2606: Fixed sort parsing of fields containing punctuation that

failed due to sort by function changes introduced in SOLR-1297
(Mitsu Hadeishi, hossman)

* SOLR-2706: contrib/clustering: The carrot.lexicalResourcesDir parameter

now works with absolute directories (Stanislaw Osinski)

* SOLR-2692: contrib/clustering: Typo in param name fixed: "carrot.fragzise"

changed to "carrot.fragSize" (Stanislaw Osinski).

* SOLR-2644: When using DIH with threads=2 the default logging is set too high
(Bill Bell via shalin)

* SOLR-2492: DIH does not commit if only deletes are processed

(James Dyer via shalin)

* SOLR-2186: DataImportHandler's multi-threaded option throws NPE

(Lance Norskog, Frank Wesemann, shalin)

* SOLR-2655: DIH multi threaded mode does not resolve attributes correctly
(Frank Wesemann, shalin)

* SOLR-2695: DIH: Documents are collected in unsynchronized list in

multi-threaded debug mode (Michael McCandless, shalin)

* SOLR-2668: DIH multithreaded mode does not rollback on errors from

EntityProcessor (Frank Wesemann, shalin)

Other Changes
----------------------

* SOLR-2629: Eliminate deprecation warnings in some JSPs.

(Bernd Fehling, hossman)

* SOLR-2743: Remove commons logging from contrib/extraction. (koji)

Build
----------------------

* SOLR-2452,SOLR-2653,LUCENE-3323,SOLR-2659,LUCENE-3329,SOLR-2666:
Rewrote the Solr build system:
 - Integrated more fully with the Lucene build system: generalized the
Lucene build system and eliminated duplication.
- Converted all Solr contribs to the Lucene/Solr conventional src/ layout:
java/, resources/, test/, and test-files/<contrib-name>.
- Created a new Solr-internal module named "core" by moving the java/,
test/, and test-files/ directories from solr/src/ to solr/core/src/.
- Merged solr/src/webapp/src/ into solr/core/src/java/.
- Eliminated solr/src/ by moving all its directories up one level;
renamed solr/src/site/ to solr/site-src/ because solr/site/ already
exists.
- Merged solr/src/common/ into solr/solrj/src/java/.
- Moved o.a.s.client.solrj.* and o.a.s.common.* tests from
solr/src/test/ to solr/solrj/src/test/.
- Made the solrj tests not depend on the solr core tests by moving
some classes from solr/src/test/ to solr/test-framework/src/java/.
- Each internal module (core/, solrj/, test-framework/, and webapp/)
now has its own build.xml, from which it is possible to run
module-specific targets. solr/build.xml delegates all build
tasks (via <ant dir="internal-module-dir"> calls) to these
modules' build.xml files.
(Steve Rowe, Robert Muir)

* LUCENE-3406: Add ant target 'package-local-src-tgz' to Lucene and Solr

to package sources from the local working copy.
(Seung-Yeoul Yang via Steve Rowe)

Documentation
----------------------

================== 3.3.0 ==================

Upgrading from Solr 3.2.0

----------------------
* SolrCore's CloseHook API has been changed in a backward-incompatible way. It
has been changed from an interface to an abstract class. Any custom
components which use the SolrCore.addCloseHook method will need to
be modified accordingly. To migrate, put your old CloseHook#close impl into
CloseHook#preClose.

New Features
----------------------

* SOLR-2378: A new, automaton-based, implementation of suggest (autocomplete)

component, offering an order of magnitude smaller memory consumption
compared to ternary trees and jaspell and very fast lookups at runtime.
(Dawid Weiss)

* SOLR-2400: Field- and DocumentAnalysisRequestHandler now provide a position

history for each token, so you can follow the token through all analysis stages.
The output contains a separate int[] attribute containing all positions from
previous Tokenizers/TokenFilters (called "positionHistory").
(Uwe Schindler)

* SOLR-2524: (SOLR-236, SOLR-237, SOLR-1773, SOLR-1311) Grouping / Field collapsing

using the Lucene grouping contrib. The search result can be grouped by field and
query.
(Martijn van Groningen, Emmanuel Keller, Shalin Shekhar Mangar, Koji Sekiguchi,
Iván de Prado, Ryan McKinley, Marc Sturlese, Peter Karich, Bojan Smid,
Charles Hornberger, Dieter Grad, Dmitry Lihachev, Doug Steigerwald,
 Karsten Sperling, Michael Gundlach, Oleg Gnatovskiy, Thomas Traeger,
Harish Agarwal, yonik, Michael McCandless, Bill Bell)

* SOLR-1331 -- Added a srcCore parameter to CoreAdminHandler's mergeindexes action

to merge one or more cores' indexes to a target core (shalin)

* SOLR-2610 -- Add an option to delete index through CoreAdmin UNLOAD action

(shalin)

* SOLR-2480: Add ignoreTikaException flag to the extraction request handler so

that users can ignore TikaException but index meta data.
(Shinichiro Abe, koji)

* SOLR-2582: Use uniqueKey for error log in UIMAUpdateRequestProcessor.

(Tommaso Teofili via koji)

Optimizations
----------------------

* SOLR-2567: Solr now defaults to TieredMergePolicy. See

http://s.apache.org/merging
for more information. (rmuir)

Bug Fixes
----------------------

* SOLR-2519: Improve text_* fieldTypes in example schema.xml: improve

cross-language defaults for text_general; break out separate
English-specific fieldTypes (Jan Høydahl, hossman, Robert Muir,
yonik, Mike McCandless)

* SOLR-2462: Fix extremely high memory usage problems with spellcheck.collate.

Separately, an additional spellcheck.maxCollationEvaluations (default=10000)
parameter is added to avoid excessive CPU time in extreme cases (e.g. long
queries with many misspelled words). (James Dyer via rmuir)

* SOLR-2579: UIMAUpdateRequestProcessor ignore error fails if text.length() < 100.

(Elmer Garduno via koji)

* SOLR-2581: UIMAToSolrMapper wrongly instantiates Type with reflection.

(Tommaso Teofili via koji)

* SOLR-2551: Check dataimport.properties for write access (if delta-import is

supported in DIH configuration) before starting an import (C S, shalin)

Other Changes
----------------------

* SOLR-2571: Add a commented out example of the spellchecker's

thresholdTokenFrequency
parameter to the example solrconfig.xml, and also add a unit test for this
feature.
(James Dyer via rmuir)

* SOLR-2576: Deprecate SpellingResult.add(Token token, int docFreq), please use

SpellingResult.addFrequency(Token token, int docFreq) instead.
(James Dyer via rmuir)

* SOLR-2574: Upgrade slf4j to v1.6.1 (shalin)

* LUCENE-3204: The maven-ant-tasks jar is now included in the source tree;
users of the generate-maven-artifacts target no longer have to manually
place this jar in the Ant classpath. NOTE: when Ant looks for the
maven-ant-tasks jar, it looks first in its pre-existing classpath, so
any copies it finds will be used instead of the copy included in the
Lucene/Solr source tree. For this reason, it is recommeded to remove
any copies of the maven-ant-tasks jar in the Ant classpath, e.g. under
~/.ant/lib/ or under the Ant installation's lib/ directory. (Steve Rowe)

* SOLR-2611: Fix typos in the example configuration (Eric Pugh via rmuir)

================== 3.2.0 ==================

Versions of Major Components
---------------------
Apache Lucene trunk
Apache Tika 0.8
Carrot2 3.4.2

Upgrading from Solr 3.1

----------------------

* The updateRequestProcessorChain for a RequestHandler is now defined

with update.chain rather than update.processor. The latter still works,
but has been deprecated.

* <uimaConfig/> just beneath <config> ... </config> is no longer supported.

It should move to UIMAUpdateRequestProcessorFactory setting.
See contrib/uima/README.txt for more details. (SOLR-2436)

Detailed Change List

----------------------

New Features
----------------------

* SOLR-2496: Add ability to specify overwrite and commitWithin as request

parameters (e.g. specified in the URL) when using the JSON update format,
and added a simplified format for specifying multiple documents.
Example: [{"id":"doc1"},{"id":"doc2"}]
(yonik)

* SOLR-2113: Add TermQParserPlugin, registered as "term". This is useful

when generating filter queries from terms returned from field faceting or
the terms component. Example: fq={!term f=weight}1.5 (hossman, yonik)

* SOLR-1915: DebugComponent now supports using a NamedList to model

Explanation objects in its responses instead of
Explanation.toString (hossman)

* SOLR-2448: Search results clustering updates: bisecting k-means

clustering algorithm added, loading of Carrot2 stop words from
<solr.home>/conf/carrot2 (SOLR-2449), using Solr's stopwords.txt
for clustering (SOLR-2450), output of cluster scores (SOLR-2505)
(Stanislaw Osinski, Dawid Weiss).

* SOLR-2503: extend UIMAUpdateRequestProcessorFactory mapping function to

map feature value to dynamicField. (koji)
* SOLR-2512: add ignoreErrors flag to UIMAUpdateRequestProcessorFactory so
that users can ignore exceptions in AE. (Tommaso Teofili, koji)

Optimizations
----------------------

Bug Fixes
----------------------

* SOLR-2445: Change the default qt to blank in form.jsp, because there is no

"standard"
request handler unless you have it in your solrconfig.xml explicitly. (koji)

* SOLR-2455: Prevent double submit of forms in admin interface.

(Jeffrey Chang via uschindler)

* SOLR-2464: Fix potential slowness in QueryValueSource (the query() function) when

the query is very sparse and may not match any documents in a segment. (yonik)

* SOLR-2469: When using java replication with replicateAfter=startup, the first

commit point on server startup is never removed. (yonik)

* SOLR-2466: SolrJ's CommonsHttpSolrServer would retry requests on failure,

regardless
of the configured maxRetries, due to HttpClient having its own retry mechanism
by default. The retryCount of HttpClient is now set to 0, and SolrJ does
the retry. (yonik)

* SOLR-2409: edismax parser - treat the text of a fielded query as a literal if the
fieldname does not exist. For example Mission: Impossible should not search on
the "Mission" field unless it's a valid field in the schema. (Ryan McKinley,
yonik)

* SOLR-2403: facet.sort=index reported incorrect results for distributed search

in a number of scenarios when facet.mincount>0. This patch also adds some
performance/algorithmic improvements when (facet.sort=count && facet.mincount=1
&& facet.limit=-1) and when (facet.sort=index && facet.mincount>0) (yonik)

* SOLR-2333: The "rename" core admin action does not persist the new name to
solr.xml
(Rasmus Hahn, Paul R. Brown via Mark Miller)

* SOLR-2390: Performance of usePhraseHighlighter is terrible on very large

Documents,
regardless of hl.maxDocCharsToAnalyze. (Mark Miller)

* SOLR-2474: The helper TokenStreams in analysis.jsp and AnalysisRequestHandlerBase

did not clear all attributes so they displayed incorrect attribute values for
tokens
in later filter stages. (uschindler, rmuir, yonik)

* SOLR-2467: Fix <analyzer class="..." /> initialization so any errors

are logged properly. (hossman)

* SOLR-2493: SolrQueryParser was fixed to not parse the SolrConfig DOM tree on each
instantiation which is a huge slowdown. (Stephane Bailliez via uschindler)

* SOLR-2495: The JSON parser could hang on corrupted input and could fail
 to detect numbers that were too large to fit in a long. (yonik)

* SOLR-2520: Make JSON response format escape \u2029 as well as \u2028

in strings since those characters are not valid in javascript strings
(although they are valid in JSON strings). (yonik)

* SOLR-2536: Add ReloadCacheRequestHandler to fix ExternalFileField bug (if

reopenReaders
set to true and no index segments have been changed, commit cannot trigger reload
external file). (koji)

* SOLR-2539: VectorValueSource.floatVal incorrectly used byteVal on sub-sources.

(Tom Liu via yonik)

* SOLR-2554: RandomSortField didn't work when used in a function query. (yonik)

Other Changes
----------------------

* SOLR-2061: Pull base tests out into a new Solr Test Framework module,
and publish binary, javadoc, and source test-framework jars.
(Drew Farris, Robert Muir, Steve Rowe)

* SOLR-2105: Rename RequestHandler param 'update.processor' to 'update.chain'.

(Jan Høydahl via Mark Miller)

* SOLR-2485: Deprecate BaseResponseWriter, GenericBinaryResponseWriter, and

GenericTextResponseWriter. These classes will be removed in 4.0. (ryan)

* SOLR-2451: Enhance assertJQ to allow individual tests to specify the

tolerance delta used in numeric equalities. This allows for slight
variance in asserting score comparisons in unit tests.
(David Smiley, Chris Hostetter)

* SOLR-2528: Remove default="true" from HtmlEncoder in example solrconfig.xml,

because html encoding confuses non-ascii users. (koji)

* SOLR-2387: add mock annotators for improved testing in contrib/uima,

(Tommaso Teofili via rmuir)

* SOLR-2436: move uimaConfig to under the uima's update processor in

solrconfig.xml. (Tommaso Teofili, koji)

Build
----------------------

* LUCENE-3006: Building javadocs will fail on warnings by default. Override with -

Dfailonjavadocwarning=false (sarowe, gsingers)

Documentation
----------------------

================== 3.1.0 ==================

Versions of Major Components
---------------------
Apache Lucene 3.1.0
Apache Tika 0.8
Carrot2 3.4.2
Velocity 1.6.1 and Velocity Tools 2.0-beta3
Apache UIMA 2.3.1-SNAPSHOT

Upgrading from Solr 1.4

----------------------

* The Lucene index format has changed and as a result, once you upgrade,
previous versions of Solr will no longer be able to read your indices.
In a master/slave configuration, all searchers/slaves should be upgraded
before the master. If the master were to be updated first, the older
searchers would not be able to read the new index format.

* The Solr JavaBin format has changed as of Solr 3.1. If you are using the
JavaBin format, you will need to upgrade your SolrJ client. (SOLR-2034)

* The experimental ALIAS command has been removed (SOLR-1637)

* Using solr.xml is recommended for single cores also (SOLR-1621)

* Old syntax of <highlighting> configuration in solrconfig.xml

is deprecated (SOLR-1696)

* The deprecated HTMLStripReader, HTMLStripWhitespaceTokenizerFactory and

HTMLStripStandardTokenizerFactory were removed. To strip HTML tags,
HTMLStripCharFilter should be used instead, and it works with any
Tokenizer of your choice. (SOLR-1657)

* Field compression is no longer supported. Fields that were formerly

compressed will be uncompressed as index segments are merged. For
shorter fields, this may actually be an improvement, as the compression
used was not very good for short text. Some indexes may get larger though.

* SOLR-1845: The TermsComponent response format was changed so that the

"terms" container is a map instead of a named list. This affects
response formats like JSON, but not XML. (yonik)

* SOLR-1876: All Analyzers and TokenStreams are now final to enforce

the decorator pattern. (rmuir, uschindler)

* LUCENE-2608: Added the ability to specify the accuracy on a per request basis.
It is recommended that implementations of SolrSpellChecker should change over to
the new SolrSpellChecker
methods using the new SpellingOptions class, but are not required to. While this
change is
backward compatible, the trunk version of Solr has already dropped support for
all but the SpellingOptions method. (gsingers)

* readercycle script was removed. (SOLR-2046)

* In previous releases, sorting or evaluating function queries on

fields that were "multiValued" (either by explicit declaration in
schema.xml or by implict behavior because the "version" attribute on
the schema was less then 1.2) did not generally work, but it would
sometimes silently act as if it succeeded and order the docs
arbitrarily. Solr will now fail on any attempt to sort, or apply a
function to, multi-valued fields
* The DataImportHandler jars are no longer included in the solr
WAR and should be added in Solr's lib directory, or referenced
via the <lib> directive in solrconfig.xml.

Detailed Change List

----------------------

New Features
----------------------

* SOLR-1302: Added several new distance based functions, including

Great Circle (haversine), Manhattan, Euclidean and String (using the
StringDistance methods in the Lucene spellchecker).
Also added geohash(), deg() and rad() convenience functions.
See https://solr.apache.org/guide/solr/latest/query-guide/function-queries.html
(gsingers)

* SOLR-1553: New dismax parser implementation (accessible as "edismax")

that supports full lucene syntax, improved reserved char escaping,
fielded queries, improved proximity boosting, and improved stopword
handling. Note: status is experimental for now. (yonik)

* SOLR-1574: Add many new functions from java Math (e.g. sin, cos) (yonik)

* SOLR-1569: Allow functions to take in literal strings by modifying the

FunctionQParser and adding LiteralValueSource (gsingers)

* SOLR-1571: Added unicode collation support though Lucene's CollationKeyFilter

(Robert Muir via shalin)

* SOLR-785: Distributed Search support for SpellCheckComponent

(Matthew Woytowitz, shalin)

* SOLR-1625: Add regexp support for TermsComponent (Uri Boness via noble)

* SOLR-1297: Add sort by Function capability (gsingers, yonik)

* SOLR-1139: Add TermsComponent Query and Response Support in SolrJ (Matt Weber via
shalin)

* SOLR-1177: Distributed Search support for TermsComponent (Matt Weber via shalin)

* SOLR-1621, SOLR-1722: Allow current single core deployments to be specified by

solr.xml (Mark Miller , noble)

* SOLR-1532: Allow StreamingUpdateSolrServer to use a provided HttpClient (Gabriele

Renzi via shalin)

* SOLR-1653: Add PatternReplaceCharFilter (koji)

* SOLR-1131: FieldTypes can now output multiple Fields per Type and still be
searched. This can be handy for hiding the details of a particular
implementation such as in the spatial case. (Chris Mattmann, shalin, noble,
gsingers, yonik)

* SOLR-1586: Add support for Geohash and Spatial Tile FieldType (Chris Mattmann,
gsingers)
* SOLR-1697: PluginInfo should load plugins w/o class attribute also (noble)

* SOLR-1268: Incorporate FastVectorHighlighter (koji)

* SOLR-1750: SolrInfoMBeanHandler added for simpler programmatic access

to info currently available from registry.jsp and stats.jsp
(ehatcher, hossman)

* SOLR-1815: SolrJ now preserves the order of facet queries. (yonik)

* SOLR-1677: Add support for choosing the Lucene Version for Lucene components
within
Solr. (Uwe Schindler, Mark Miller)

* SOLR-1379: Add RAMDirectoryFactory for non-persistent in memory index storage.

(Alex Baranov via yonik)

* SOLR-1857: Synced Solr analysis with Lucene 3.1. Added KeywordMarkerFilterFactory

and StemmerOverrideFilterFactory, which can be used to tune stemming algorithms.
Added factories for Bulgarian, Czech, Hindi, Turkish, and Wikipedia analysis.
Improved the
performance of SnowballPorterFilterFactory. (rmuir)

* SOLR-1657: Converted remaining TokenStreams to the Attributes-based API. All Solr

TokenFilters now support custom Attributes, and some have improved performance:
especially WordDelimiterFilter and CommonGramsFilter. (rmuir, cmale, uschindler)

* SOLR-1740: ShingleFilterFactory supports the "minShingleSize" and

"tokenSeparator"
parameters for controlling the minimum shingle size produced by the filter, and
the separator string that it uses, respectively. (Steven Rowe via rmuir)

* SOLR-744: ShingleFilterFactory supports the "outputUnigramsIfNoShingles"

parameter, to output unigrams if the number of input tokens is fewer than
minShingleSize, and no shingles can be generated.
(Chris Harris via Steven Rowe)

* SOLR-1923: PhoneticFilterFactory now has support for the

Caverphone algorithm. (rmuir)

* SOLR-1957: The VelocityResponseWriter contrib moved to core.

Example search UI now available at http://localhost:8983/solr/browse
(ehatcher)

* SOLR-1974: Add LimitTokenCountFilterFactory. (koji)

* SOLR-1966: QueryElevationComponent can now return just the included results in

the elevation file (gsingers, yonik)

* SOLR-1556: TermVectorComponent now supports per field overrides. Also, it now

throws an error
if passed in fields do not exist and warnings
if fields that do not have term vector options (termVectors, offsets, positions)
that align with the schema declaration. It also
will now return warnings about (gsingers)

* SOLR-1985: FastVectorHighlighter: add wrapper class for Lucene's

SingleFragListBuilder (koji)
* SOLR-1984: Add HyphenationCompoundWordTokenFilterFactory. (PB via rmuir)

* SOLR-397: Date Faceting now supports a "facet.date.include" param

for specifying when the upper & lower end points of computed date
ranges should be included in the range. Legal values are: "all",
"lower", "upper", "edge", and "outer". For backwards compatibility
the default value is the set: [lower,upper,edge], so that all ranges
between start and end are inclusive of their endpoints, but the
"before" and "after" ranges are not.

* SOLR-945: JSON update handler that accepts add, delete, commit

commands in JSON format. (Ryan McKinley, yonik)

* SOLR-2015: Add a boolean attribute autoGeneratePhraseQueries to TextField.

autoGeneratePhraseQueries="true" (the default) causes the query parser to
generate phrase queries if multiple tokens are generated from a single
non-quoted analysis string. For example WordDelimiterFilter splitting text:pdp-
11
will cause the parser to generate text:"pdp 11" rather than (text:PDP OR
text:11).
Note that autoGeneratePhraseQueries="true" tends to not work well for non
whitespace
delimited languages. (yonik)

* SOLR-1925: Add CSVResponseWriter (use wt=csv) that returns the list of documents
in CSV format. (Chris Mattmann, yonik)

* SOLR-1240: "Range Faceting" has been added. This is a generalization

of the existing "Date Faceting" logic so that it now supports any
all stock numeric field types that support range queries in addition
to dates. facet.date is now deprecated in favor of this generalized mechanism.
(Gijs Kunze, hossman)

* SOLR-2021: Add SolrEncoder plugin to Highlighter. (koji)

* SOLR-2030: Make FastVectorHighlighter use of SolrEncoder. (koji)

* SOLR-2053: Add support for custom comparators in Solr spellchecker, per LUCENE-
2479 (gsingers)

* SOLR-2049: Add hl.multiValuedSeparatorChar for FastVectorHighlighter, per LUCENE-

2603. (koji)

* SOLR-2059: Add "types" attribute to WordDelimiterFilterFactory, which

allows you to customize how WordDelimiterFilter tokenizes text with
a configuration file. (Peter Karich, rmuir)

* SOLR-2099: Add ability to throttle rsync based replication using rsync option --
bwlimit.
(Brandon Evans via koji)

* SOLR-1316: Create autosuggest component.

(Ankul Garg, Jason Rutherglen, Shalin Shekhar Mangar, Grant Ingersoll, Robert
Muir, ab)

* SOLR-1568: Added "native" filtering support for PointType, GeohashField. Added

LatLonType with filtering support too. See
http://wiki.apache.org/solr/SpatialSearch and the example. Refactored some items
in Lucene spatial.
Removed SpatialTileField as the underlying CartesianTier is broken beyond repair
and is going to be moved. (gsingers)

* SOLR-2128: Full parameter substitution for function queries.

Example: q=add($v1,$v2)&v1=mul(popularity,5)&v2=20.0
(yonik)

* SOLR-2133: Function query parser can now parse multiple comma separated
value sources. It also now fails if there is extra unexpected text
after parsing the functions, instead of silently ignoring it.
This allows expressions like q=dist(2,vector(1,2),$pt)&pt=3,4 (yonik)

* SOLR-2157: Suggester should return alpha-sorted results when

onlyMorePopular=false (ab)

* SOLR-2010: Added ability to verify that spell checking collations have

actual results in the index. (James Dyer via gsingers)

* SOLR-2188: Added "maxTokenLength" argument to the factories for ClassicTokenizer,

StandardTokenizer, and UAX29URLEmailTokenizer. (Steven Rowe)

* SOLR-2129: Added a Solr module for dynamic metadata extraction/indexing with

Apache UIMA.
See contrib/uima/README.txt for more information. (Tommaso Teofili via rmuir)

* SOLR-2325: Allow tagging and exclusion of main query for faceting. (yonik)

* SOLR-2263: Add ability for RawResponseWriter to stream binary files as well as

text files. (Eric Pugh via yonik)

* SOLR-860: Add debug output for MoreLikeThis. (koji)

* SOLR-1057: Add PathHierarchyTokenizerFactory. (ryan, koji)

* SOLR-1804: Re-enabled clustering component on trunk, updated to latest

version of Carrot2. No more LGPL run-time dependencies. This release of
C2 also does not have a specific Lucene dependency.
(Stanislaw Osinski, gsingers)

* SOLR-2282: Add distributed search support for search result clustering.

(Brad Giaccio, Dawid Weiss, Stanislaw Osinski, rmuir, koji)

* SOLR-2210: Add icu-based tokenizer and filters to contrib/analysis-extras (rmuir)

* SOLR-1336: Add SmartChinese (word segmentation for Simplified Chinese)

tokenizer and filters to contrib/analysis-extras (rmuir)

* SOLR-2211,LUCENE-2763: Added UAX29URLEmailTokenizerFactory, which implements

UAX#29, a unicode algorithm with good results for most languages, as well as
URL and E-mail tokenization according to the relevant RFCs.
(Tom Burton-West via rmuir)

* SOLR-2237: Added StempelPolishStemFilterFactory to contrib/analysis-extras

(rmuir)

* SOLR-1525: allow DIH to refer to core properties (noble)

* SOLR-1547: DIH TemplateTransformer copy objects more intelligently when the

 template is a single variable (noble)

* SOLR-1627: DIH VariableResolver should be fetched just in time (noble)

* SOLR-1583: DIH Create DataSources that return InputStream (noble)

* SOLR-1358: Integration of Tika and DataImportHandler (Akshay Ukey, noble)

* SOLR-1654: TikaEntityProcessor example added DIHExample

(Akshay Ukey via noble)

* SOLR-1678: Move onError handling to DIH framework (noble)

* SOLR-1352: Multi-threaded implementation of DIH (noble)

* SOLR-1721: Add explicit option to run DataImportHandler in synchronous mode

(Alexey Serba via noble)

* SOLR-1737: Added FieldStreamDataSource (noble)

Optimizations
----------------------

* SOLR-1679: Don't build up string messages in SolrCore.execute unless they

are necessary for the current log level.
(Fuad Efendi and hossman)

* SOLR-1874: Optimize PatternReplaceFilter for better performance. (rmuir,

uschindler)

* SOLR-1968: speed up initial filter cache population for facet.method=enum and

also big terms for multi-valued facet.method=fc. The resulting speedup
for the first facet request is anywhere from 30% to 32x, depending on how many
terms are in the field and how many documents match per term. (yonik)

* SOLR-2089: Speed up UnInvertedField faceting (facet.method=fc for

multi-valued fields) when facet.limit is both high, and a high enough
percentage of the number of unique terms in the field. Extreme cases
yield speedups over 3x. (yonik)

* SOLR-2046: add common functions to scripts-util. (koji)

* SOLR-1684: Switch clustering component to use the

SolrIndexSearcher.doc(int, Set<String>) method b/c it can use the document
cache (gsingers)

* SOLR-2200: Improve the performance of DataImportHandler for large

delta-import updates. (Mark Waddle via rmuir)

Bug Fixes
----------------------
* SOLR-1769: Solr 1.4 Replication - Repeater throwing NullPointerException (Jörgen
Rydenius via noble)

* SOLR-1432: Make the new ValueSource.getValues(context,reader) delegate

to the original ValueSource.getValues(reader) so custom sources
will work. (yonik)
* SOLR-1572: FastLRUCache correctly implemented the LRU policy only
for the first 2B accesses. (yonik)

* SOLR-1582: copyField was ignored for BinaryField types (gsingers)

* SOLR-1563: Binary fields, including trie-based numeric fields, caused null

pointer exceptions in the luke request handler. (yonik)

* SOLR-1577: The example solrconfig.xml defaulted to a solr data dir

relative to the current working directory, even if a different solr home
was being used. The new behavior changes the default to a zero length
string, which is treated the same as if no dataDir had been specified,
hence the "data" directory under the solr home will be used. (yonik)

* SOLR-1584: SolrJ - SolrQuery.setIncludeScore() incorrectly added

fl=score to the parameter list instead of appending score to the
existing field list. (yonik)

* SOLR-1580: Solr Configuration ignores 'mergeFactor' parameter, always

uses Lucene default. (Lance Norskog via Mark Miller)

* SOLR-1593: ReverseWildcardFilter didn't work for surrogate pairs

(i.e. code points outside of the BMP), resulting in incorrect
matching. This change requires reindexing for any content with
such characters. (Robert Muir, yonik)

* SOLR-1596: A rollback operation followed by the shutdown of Solr

or the close of a core resulted in a warning:
"SEVERE: SolrIndexWriter was not closed prior to finalize()" although
there were no other consequences. (yonik)

* SOLR-1595: StreamingUpdateSolrServer used the platform default character

set when streaming updates, rather than using UTF-8 as the HTTP headers
indicated, leading to an encoding mismatch. (hossman, yonik)

* SOLR-1587: A distributed search request with fl=score, didn't match

the behavior of a non-distributed request since it only returned
the id,score fields instead of all fields in addition to score. (yonik)

* SOLR-1601: Schema browser does not indicate presence of charFilter. (koji)

* SOLR-1615: Backslash escaping did not work in quoted strings

for local param arguments. (Wojtek Piaseczny, yonik)

* SOLR-1628: log contains incorrect number of adds and deletes.

(Thijs Vonk via yonik)

* SOLR-343: Date faceting now respects facet.mincount limiting

(Uri Boness, Raiko Eckstein via hossman)

* SOLR-1624: Highlighter only highlights values from the first field value
in a multivalued field when term positions (term vectors) are stored.
(Chris Harris via yonik)

* SOLR-1635: Fixed error message when numeric values can't be parsed by

DOMUtils - notably for plugin init params in solrconfig.xml.
(hossman)

* SOLR-1651: Fixed Incorrect dataimport handler package name in SolrResourceLoader

 (Akshay Ukey via shalin)

* SOLR-1660: CapitalizationFilter crashes if you use the maxWordCountOption

(Robert Muir via shalin)

* SOLR-1667: PatternTokenizer does not reset attributes such as

positionIncrementGap
(Robert Muir via shalin)

* SOLR-1711: SolrJ - StreamingUpdateSolrServer had a race condition that

could halt the streaming of documents. The original patch to fix this
(never officially released) introduced another hanging bug due to
connections not being released.
(Attila Babo, Erik Hetzner, Johannes Tuchscherer via yonik)

* SOLR-1748, SOLR-1747, SOLR-1746, SOLR-1745, SOLR-1744: Streams and Readers

retrieved from ContentStreams are not closed in various places, resulting
in file descriptor leaks.
(Christoff Brill, Mark Miller)

* SOLR-1753: StatsComponent throws NPE when getting statistics for facets in

distributed search
(Janne Majaranta via koji)

* SOLR-1736:In the slave , If 'mov'ing file does not succeed , copy the file
(noble)

* SOLR-1579: Fixes to XML escaping in stats.jsp

(David Bowen and hossman)

* SOLR-1777: fieldTypes with sortMissingLast=true or sortMissingFirst=true can

result in incorrectly sorted results. (yonik)

* SOLR-1798: Small memory leak (~100 bytes) in fastLRUCache for every

commit. (yonik)

* SOLR-1823: Fixed XMLResponseWriter (via XMLWriter) so it no longer throws

a ClassCastException when a Map containing a non-String key is used.
(Frank Wesemann, hossman)

* SOLR-1797: fix ConcurrentModificationException and potential memory

leaks in ResourceLoader. (yonik)

* SOLR-1850: change KeepWordFilter so a new word set is not created for

each instance (John Wang via yonik)

* SOLR-1706: fixed WordDelimiterFilter for certain combinations of options

where it would output incorrect tokens. (Robert Muir, Chris Male)

* SOLR-1936: The JSON response format needed to escape unicode code point
U+2028 - 'LINE SEPARATOR' (Robert Hofstra, yonik)

* SOLR-1914: Change the JSON response format to output float/double

values of NaN,Infinity,-Infinity as strings. (yonik)

* SOLR-1948: PatternTokenizerFactory should use parent's args (koji)

* SOLR-1870: Indexing documents using the 'javabin' format no longer

fails with a ClassCastException whenSolrInputDocuments contain field
 values which are Collections or other classes that implement
Iterable. (noble, hossman)

* SOLR-1981: Solr will now fail correctly if solr.xml attempts to

specify multiple cores that have the same name (hossman)

* SOLR-1791: Fix messed up core names on admin gui (yonik via koji)

* SOLR-1995: Change date format from "hour in am/pm" to "hour in day"

in CoreContainer and SnapShooter. (Hayato Ito, koji)

* SOLR-2008: avoid possible RejectedExecutionException w/autoCommit

by making SolreCore close the UpdateHandler before closing the
SearchExecutor. (NarasimhaRaju, hossman)

* SOLR-2036: Avoid expensive fieldCache ram estimation for the

admin stats page. (yonik)

* SOLR-2047: ReplicationHandler should accept bool type for enable flag. (koji)

* SOLR-1630: Fix spell checking collation issue related to token positions (rmuir,
gsingers)

* SOLR-2100: The replication handler backup command didn't save the commit
point and hence could fail when a newer commit caused the older commit point
to be removed before it was finished being copied. This did not affect
normal master/slave replication. (Peter Sturge via yonik)

* SOLR-2114: Fixed parsing error in hsin function. The function signature has
changed slightly. (gsingers)

* SOLR-2083: SpellCheckComponent misreports suggestions when distributed (James

Dyer via gsingers)

* SOLR-2111: Change exception handling in distributed faceting to work more

like non-distributed faceting, change facet_counts/exception from a String
to a List<String> to enable listing all exceptions that happened, and
prevent an exception in one facet command from affecting another
facet command. (yonik)

* SOLR-2110: Remove the restriction on names for local params

substitution/dereferencing. Properly encode local params in
distributed faceting. (yonik)

* SOLR-2135: Fix behavior of ConcurrentLRUCache when asking for

getLatestAccessedItems(0) or getOldestAccessedItems(0).
(David Smiley via hossman)

* SOLR-2148: Highlighter doesn't support q.alt. (koji)

* SOLR-2180: It was possible for EmbeddedSolrServer to leave searchers

open if a request threw an exception. (yonik)

* SOLR-2173: Suggester should always rebuild Lookup data if Lookup.load fails. (ab)

* SOLR-2081: BaseResponseWriter.isStreamingDocs causes

SingleResponseWriter.end to be called 2x
(Chris A. Mattmann via hossman)
* SOLR-2219: The init() method of every SolrRequestHandler was being
called twice. (ambikeshwar singh and hossman)

* SOLR-2285: duplicate SolrEventListeners no longer created (hossman)

* SOLR-1993: fix String cast assumption in JavaBinCodec - specific

addresses "commitWithin" option on Update requests.
(noble, hossman, and Maxim Valyanskiy)

* SOLR-2261: fix velocity template layout.vm that referred to an older

version of jquery. (Eric Pugh via rmuir)

* SOLR-2307: fix bug in PHPSerializedResponseWriter (wt=phps) when

dealing with SolrDocumentList objects -- ie: sharded queries.
(Antonio Verni via hossman)

* SOLR-2127: Fixed serialization of default core and indentation of solr.xml when

serializing.
(Ephraim Ofir, Mark Miller)

* SOLR-2320: Fixed ReplicationHandler detail reporting for masters

(hossman)

* SOLR-482: Provide more exception handling in CSVLoader (gsingers)

* SOLR-1283: HTMLStripCharFilter sometimes threw a "Mark Invalid" exception.

(Julien Coloos, hossman, yonik)

* SOLR-2085: Improve SolrJ behavior when FacetComponent comes before

QueryComponent (Tomas Salfischberger via hossman)

* SOLR-1940: Fix SolrDispatchFilter behavior when Content-Type is

unknown (Lance Norskog and hossman)

* SOLR-1983: snappuller fails when modifiedConfFiles is not empty and

full copy of index is needed. (Alexander Kanarsky via yonik)

* SOLR-2156: SnapPuller fails to clean Old Index Directories on Full Copy

(Jayendra Patil via yonik)

* SOLR-96: Fix XML parsing in XMLUpdateRequestHandler and

DocumentAnalysisRequestHandler to respect charset from XML file and only
use HTTP header's "Content-Type" as a "hint". (uschindler)

* SOLR-2339: Fix sorting to explicitly generate an error if you

attempt to sort on a multiValued field. (hossman)

* SOLR-2348: Fix field types to explicitly generate an error if you

attempt to get a ValueSource for a multiValued field. (hossman)

* SOLR-2380: Distributed faceting could miss values when facet.sort=index

and when facet.offset was greater than 0. (yonik)

* SOLR-1656: XIncludes and other HREFs in XML files loaded by ResourceLoader

are fixed to be resolved using the URI standard (RFC 2396). The system
identifier is no longer a plain filename with path, it gets initialized
using a custom URI scheme "solrres:". This scheme is resolved using a
EntityResolver that utilizes ResourceLoader
(org.apache.solr.common.util.SystemIdResolver). This makes all relative
 pathes in Solr's config files behave like expected. This change
introduces some backwards breaks in the API: Some config classes
(Config, SolrConfig, IndexSchema) were changed to take
org.xml.sax.InputSource instead of InputStream. There may also be some
backwards breaks in existing config files, it is recommended to check
your config files / XSLTs and replace all XIncludes/HREFs that were
hacked to use absolute paths to use relative ones. (uschindler)

* SOLR-309: Fix FieldType so setting an analyzer on a FieldType that

doesn't expect it will generate an error. Practically speaking this
means that Solr will now correctly generate an error on
initialization if the schema.xml contains an analyzer configuration
for a fieldType that does not use TextField. (hossman)

* SOLR-2192: StreamingUpdateSolrServer.blockUntilFinished was not

thread safe and could throw an exception. (yonik)

* SOLR-1692: Fix bug in clustering component relating to carrot.produceSummary

option (gsingers)

* SOLR-1756: The date.format setting for extraction request handler causes

ClassCastException when enabled and the config code that parses this setting
does not properly use the same iterator instance.
(Christoph Brill, Mark Miller)

* SOLR-1638: Fixed NullPointerException during DIH import if uniqueKey is not

specified in schema (Akshay Ukey via shalin)

* SOLR-1639: Fixed misleading error message when dataimport.properties is not

writable (shalin)

* SOLR-1598: DIH: Reader used in PlainTextEntityProcessor is not explicitly

closed (Sascha Szott via noble)

* SOLR-1759: DIH: $skipDoc was not working correctly

(Gian Marco Tagliani via noble)

* SOLR-1762: DIH: DateFormatTransformer does not work correctly with

non-default locale dates (tommy chheng via noble)

* SOLR-1757: DIH multithreading sometimes throws NPE (noble)

* SOLR-1766: DIH with threads enabled doesn't respond to the abort command
(Michael Henson via noble)

* SOLR-1767: dataimporter.functions.escapeSql() does not escape backslash

character (Sean Timm via noble)

* SOLR-1811: formatDate should use the current NOW value always

(Sean Timm via noble)

* SOLR-1794: Dataimport of CLOB fields fails when getCharacterStream() is

defined in a superclass. (Gunnar Gauslaa Bergem via rmuir)

* SOLR-2057: DataImportHandler never calls UpdateRequestProcessor.finish()

(Drew Farris via koji)

* SOLR-1973: Empty fields in XML update messages confuse DataImportHandler.

(koji)
* SOLR-2221: Use StrUtils.parseBool() to get values of boolean options in DIH.
true/on/yes (for TRUE) and false/off/no (for FALSE) can be used for
sub-options (debug, verbose, synchronous, commit, clean, optimize) for
full/delta-import commands. (koji)

* SOLR-2310: DIH: getTimeElapsedSince() returns incorrect hour value when

the elapse is over 60 hours (tom liu via koji)

* SOLR-2252: DIH: When a child entity in nested entities is rootEntity="true",

delta-import doesn't work. (koji)

* SOLR-2330: solrconfig.xml files in example-DIH are broken. (Matt Parker, koji)

* SOLR-1191: resolve DataImportHandler deltaQuery column against pk when pk

has a prefix (e.g. pk="book.id" deltaQuery="select id from ..."). More
useful error reporting when no match found (previously failed with a
NullPointerException in log and no clear user feedback). (gthb via yonik)

* SOLR-2116: Fix TikaConfig classloader bug in TikaEntityProcessor

(Martijn van Groningen via hossman)

Other Changes
----------------------

* SOLR-1602: Refactor SOLR package structure to include o.a.solr.response

and move QueryResponseWriters in there
(Chris A. Mattmann, ryan, hoss)

* SOLR-1516: Addition of an abstract BaseResponseWriter class to simplify the

development of QueryResponseWriter implementations.
(Chris A. Mattmann via noble)

* SOLR-1592: Refactor XMLWriter startTag to allow arbitrary attributes to be

written
(Chris A. Mattmann via noble)

* SOLR-1561: Added Lucene 2.9.1 spatial contrib jar to lib. (gsingers)

* SOLR-1570: Log warnings if uniqueKey is multi-valued or not stored (hossman,

shalin)

* SOLR-1558: QueryElevationComponent only works if the uniqueKey field is

implemented using StrField. In previous versions of Solr no warning or
error would be generated if you attempted to use QueryElevationComponent,
it would just fail in unexpected ways. This has been changed so that it
will fail with a clear error message on initialization. (hossman)

* SOLR-1611: Added Lucene 2.9.1 collation contrib jar to lib (shalin)

* SOLR-1608: Extract base class from TestDistributedSearch to make

it easy to write test cases for other distributed components. (shalin)

* Upgraded to Lucene 2.9-dev r888785 (shalin)

* SOLR-1610: Generify SolrCache (Jason Rutherglen via shalin)

* SOLR-1637: Remove ALIAS command

* SOLR-1662: Added Javadocs in BufferedTokenStream and fixed incorrect cloning
in TestBufferedTokenStream (Robert Muir, Uwe Schindler via shalin)

* SOLR-1674: Improve analysis tests and cut over to new TokenStream API.
(Robert Muir via Mark Miller)

* SOLR-1661: Remove adminCore from CoreContainer . removed deprecated methods

setAdminCore(), getAdminCore() (noble)

* SOLR-1704: Google collections moved from clustering to core (noble)

* SOLR-1268: Add Lucene 2.9-dev r888785 FastVectorHighlighter contrib jar to lib.

(koji)

* SOLR-1538: Reordering of object allocations in ConcurrentLRUCache to eliminate

(an extremely small) potential for deadlock.
(gabriele renzi via hossman)

* SOLR-1588: Removed some very old dead code.

(Chris A. Mattmann via hossman)

* SOLR-1696 : Deprecate old <highlighting> syntax and move configuration to

HighlightComponent (noble)

* SOLR-1727: SolrEventListener should extend NamedListInitializedPlugin (noble)

* SOLR-1771: Improved error message when StringIndex cannot be initialized

for a function query (hossman)

* SOLR-1695: Improved error messages when adding a document that does not
contain exactly one value for the uniqueKey field (hossman)

* SOLR-1776: DismaxQParser and ExtendedDismaxQParser now use the schema.xml

"defaultSearchField" as the default value for the "qf" param instead of failing
with an error when "qf" is not specified. (hossman)

* SOLR-1851: luceneAutoCommit no longer has any effect - it has been remove (Mark
Miller)

* SOLR-1865: SolrResourceLoader.getLines ignores Byte Order Markers (BOMs) at the

beginning of input files, these are often created by editors such as Windows
Notepad. (rmuir, hossman)

* SOLR-1938: ElisionFilterFactory will use a default set of French contractions

if you do not supply a custom articles file. (rmuir)

* SOLR-2003: SolrResourceLoader will report any encoding errors, rather than

silently using replacement characters for invalid inputs (blargy via rmuir)

* SOLR-1804: Google collections updated to Google Guava (which is a superset of

collections and contains bug fixes) (gsingers)

* SOLR-2034: Switch to JavaBin codec version 2. Strings are now serialized

as the number of UTF-8 bytes, followed by the bytes in UTF-8. Previously
Strings were serialized as the number of UTF-16 chars, followed by the
bytes in Modified UTF-8. (hossman, yonik, rmuir)

* SOLR-2013: Add mapping-FoldToASCII.txt to example conf directory.

(Steven Rowe via koji)
* SOLR-2213: Upgrade to jQuery 1.4.3 (Erick Erickson via ryan)

* SOLR-1826: Add unit tests for highlighting with termOffsets=true

and overlapping tokens. (Stefan Oestreicher via rmuir)

* SOLR-2340: Add version infos to message in JavaBinCodec when throwing

exception. (koji)

* SOLR-2350: Since Solr no longer requires XML files to be in UTF-8

(see SOLR-96) SimplePostTool (aka: post.jar) has been improved to
work with files of any mime-type or charset. (hossman)

* SOLR-2365: Move DIH jars out of solr.war (David Smiley via yonik)

* SOLR-2381: Include a patched version of Jetty (6.1.26 + JETTY-1340)

to fix problematic UTF-8 handling for supplementary characters.
(Bernd Fehling, uschindler, yonik, rmuir)

* SOLR-2391: The preferred Content-Type for XML was changed to

application/xml. XMLResponseWriter now only delivers using this
type; updating documents and analyzing documents is still supported
using text/xml as Content-Type, too. If you have clients that are
hardcoded on text/xml as Content-Type, you have to change them.
(uschindler, rmuir)

* SOLR-2414: All ResponseWriters now use only ServletOutputStreams

and wrap their own Writer around it when serializing. This fixes
the bug in PHPSerializedResponseWriter that produced wrong string
length if the servlet container had a broken UTF-8 encoding that was
in fact CESU-8 (see SOLR-1091). The system property to enable the
CESU-8 byte counting in PHPSerializesResponseWriters for broken
servlet containers was therefore removed and is now ignored if set.
Output is always UTF-8. (uschindler, yonik, rmuir)

* SOLR-141: Errors and Exceptions are formated by ResponseWriter.

(Mike Sokolov, Rich Cariens, Daniel Naber, ryan)

* SOLR-1902: Upgraded to Tika 0.8 and changed deprecated parse call

* SOLR-1813: Add ICU4j to contrib/extraction libs and add tests for Arabic
extraction (Robert Muir via gsingers)

* SOLR-1821: Fix TimeZone-dependent test failure in TestEvaluatorBag.

(Chris Male via rmuir)

* SOLR-2367: Reduced noise in test output by ensuring the properties file

can be written. (Gunnlaugur Thor Briem via rmuir)

Build
----------------------

* SOLR-1522: Automated release signing process. (gsingers)

* SOLR-1891: Make lucene-jars-to-solr fail if copying any of the jars fails, and
update clean to remove the jars in that directory (Mark Miller)

* LUCENE-2466: Commons-Codec was upgraded from 1.3 to 1.4. (rmuir)

* SOLR-2042: Fixed some Maven deps (Drew Farris via gsingers)

* LUCENE-2657: Switch from using Maven POM templates to full POMs when
generating Maven artifacts (Steven Rowe)

Documentation
----------------------

* SOLR-1590: Javadoc for XMLWriter#startTag

(Chris A. Mattmann via hossman)

* SOLR-1792: Documented peculiar behavior of TestHarness.LocalRequestFactory

(hossman)

================== Release 1.4.1 ==================

Release Date: See http://lucene.apache.org/solr for the official release date.

Upgrading from Solr 1.4

-----------------------

This is a bug fix release - no changes are required when upgrading from Solr 1.4.
However, a reindex is needed for some of the analysis fixes to take effect.

Versions of Major Components

----------------------------
Apache Lucene 2.9.3
Apache Tika 0.4
Carrot2 3.1.0

Lucene Information
----------------

Since Solr is built on top of Lucene, many people add customizations to Solr
that are dependent on Lucene. Please see http://lucene.apache.org/java/2_9_3/,
especially http://lucene.apache.org/java/2_9_3/changes/Changes.html for more
information on the version of Lucene used in Solr.

Bug Fixes
----------------------

* SOLR-1934: Upgrade to Apache Lucene 2.9.3 to obtain several bug

fixes from the previous 2.9.1. See the Lucene 2.9.3 release notes
for details. (hossman, Mark Miller)

* SOLR-1432: Make the new ValueSource.getValues(context,reader) delegate

to the original ValueSource.getValues(reader) so custom sources
will work. (yonik)

* SOLR-1572: FastLRUCache correctly implemented the LRU policy only

for the first 2B accesses. (yonik)

* SOLR-1595: StreamingUpdateSolrServer used the platform default character

set when streaming updates, rather than using UTF-8 as the HTTP headers
indicated, leading to an encoding mismatch. (hossman, yonik)

* SOLR-1660: CapitalizationFilter crashes if you use the maxWordCountOption

(Robert Muir via shalin)

* SOLR-1662: Added Javadocs in BufferedTokenStream and fixed incorrect cloning

 in TestBufferedTokenStream (Robert Muir, Uwe Schindler via shalin)

* SOLR-1711: SolrJ - StreamingUpdateSolrServer had a race condition that

could halt the streaming of documents. The original patch to fix this
(never officially released) introduced another hanging bug due to
connections not being released. (Attila Babo, Erik Hetzner via yonik)

* SOLR-1748, SOLR-1747, SOLR-1746, SOLR-1745, SOLR-1744: Streams and Readers

retrieved from ContentStreams are not closed in various places, resulting
in file descriptor leaks.
(Christoff Brill, Mark Miller)

* SOLR-1580: Solr Configuration ignores 'mergeFactor' parameter, always

uses Lucene default. (Lance Norskog via Mark Miller)

* SOLR-1777: fieldTypes with sortMissingLast=true or sortMissingFirst=true can

result in incorrectly sorted results. (yonik)

* SOLR-1797: fix ConcurrentModificationException and potential memory

leaks in ResourceLoader. (yonik)

* SOLR-1798: Small memory leak (~100 bytes) in fastLRUCache for every

commit. (yonik)

* SOLR-1522: Show proper message if <script> tag is missing for DIH

ScriptTransformer (noble)

* SOLR-1538: Reordering of object allocations in ConcurrentLRUCache to eliminate

(an extremely small) potential for deadlock.
(gabriele renzi via hossman)

* SOLR-1558: QueryElevationComponent only works if the uniqueKey field is

implemented using StrField. In previous versions of Solr no warning or
error would be generated if you attempted to use QueryElevationComponent,
it would just fail in unexpected ways. This has been changed so that it
will fail with a clear error message on initialization. (hossman)

* SOLR-1563: Binary fields, including trie-based numeric fields, caused null

pointer exceptions in the luke request handler. (yonik)

* SOLR-1579: Fixes to XML escaping in stats.jsp

(David Bowen and hossman)

* SOLR-1582: copyField was ignored for BinaryField types (gsingers)

* SOLR-1596: A rollback operation followed by the shutdown of Solr

or the close of a core resulted in a warning:
"SEVERE: SolrIndexWriter was not closed prior to finalize()" although
there were no other consequences. (yonik)

* SOLR-1651: Fixed Incorrect dataimport handler package name in SolrResourceLoader

(Akshay Ukey via shalin)

* SOLR-1936: The JSON response format needed to escape unicode code point
U+2028 - 'LINE SEPARATOR' (Robert Hofstra, yonik)

* SOLR-1852: Fix WordDelimiterFilterFactory bug where position increments

were not being applied properly to subwords. (Peter Wolanin via Robert Muir)
* SOLR-1706: fixed WordDelimiterFilter for certain combinations of options
where it would output incorrect tokens. (Robert Muir, Chris Male)

* SOLR-1948: PatternTokenizerFactory should use parent's args (koji)

* SOLR-1870: Indexing documents using the 'javabin' format no longer

fails with a ClassCastException whenSolrInputDocuments contain field
values which are Collections or other classes that implement
Iterable. (noble, hossman)

* SOLR-1769 Solr 1.4 Replication - Repeater throwing NullPointerException (noble)

================== Release 1.4.0 ==================

Release Date: See http://lucene.apache.org/solr for the official release date.

Upgrading from Solr 1.3

-----------------------

There is a new default faceting algorithm for multiVaued fields that should be
faster for most cases. One can revert to the previous algorithm (which has
also been improved somewhat) by adding facet.method=enum to the request.

Searching and sorting is now done on a per-segment basis, meaning that

the FieldCache entries used for sorting and for function queries are
created and used per-segment and can be reused for segments that don't
change between index updates. While generally beneficial, this can lead
to increased memory usage over 1.3 in certain scenarios:
1) A single valued field that was used for both sorting and faceting
in 1.3 would have used the same top level FieldCache entry. In 1.4,
sorting will use entries at the segment level while faceting will still
use entries at the top reader level, leading to increased memory usage.
2) Certain function queries such as ord() and rord() require a top level
FieldCache instance and can thus lead to increased memory usage. Consider
replacing ord() and rord() with alternatives, such as function queries
based on ms() for date boosting.

If you use custom Tokenizer or TokenFilter components in a chain specified in

schema.xml, they must support reusability. If your Tokenizer or TokenFilter
maintains state, it should implement reset(). If your TokenFilteFactory does
not return a subclass of TokenFilter, then it should implement reset() and call
reset() on its input TokenStream. TokenizerFactory implementations must
now return a Tokenizer rather than a TokenStream.

New users of Solr 1.4 will have omitTermFreqAndPositions enabled for non-text
indexed fields by default, which avoids indexing term frequency, positions, and
payloads, making the index smaller and faster. If you are upgrading from an
earlier Solr release and want to enable omitTermFreqAndPositions by default,
change the schema version from 1.1 to 1.2 in schema.xml. Remove any existing
index and restart Solr to ensure that omitTermFreqAndPositions completely takes
affect.

The default QParserPlugin used by the QueryComponent for parsing the "q" param
has been changed, to remove support for the deprecated use of ";" as a separator
between the query string and the sort options when no "sort" param was used.
Users who wish to continue using the semi-colon based method of specifying the
sort options should explicitly set the defType param to "lucenePlusSort" on all
requests. (The simplest way to do this is by specifying it as a default param
for your request handlers in solrconfig.xml, see the example solrconfig.xml for
sample syntax.)
If spellcheck.extendedResults=true, the response format for suggestions
has changed, see SOLR-1071.

Use of the "charset" option when configuring the following Analysis

Factories has been deprecated and will cause a warning to be logged.
In future versions of Solr attempting to use this option will cause an
error. See SOLR-1410 for more information.
- GreekLowerCaseFilterFactory
- RussianStemFilterFactory
- RussianLowerCaseFilterFactory
- RussianLetterTokenizerFactory

DIH: Evaluator API has been changed in a non back-compatible way. Users who
have developed custom Evaluators will need to change their code according to
the new API for it to work. See SOLR-996 for details.

DIH: The formatDate evaluator's syntax has been changed. The new syntax is
formatDate(<variable>, '<format_string>'). For example,
formatDate(x.date, 'yyyy-MM-dd'). In the old syntax, the date string was
written without a single-quotes. The old syntax has been deprecated and will
be removed in 1.5, until then, using the old syntax will log a warning.

DIH: The Context API has been changed in a non back-compatible way. In
particular, the Context.currentProcess() method now returns a String
describing the type of the current import process instead of an int.
Similarily, the public constants in Context viz. FULL_DUMP, DELTA_DUMP and
FIND_DELTA are changed to a String type. See SOLR-969 for details.

DIH: The EntityProcessor API has been simplified by moving logic for applying
transformers and handling multi-row outputs from Transformers into an
EntityProcessorWrapper class. The EntityProcessor#destroy is now called once
per parent-row at the end of row (end of data). A new method
EntityProcessor#close is added which is called at the end of import.

DIH: In Solr 1.3, if the last_index_time was not available (first import) and
a delta-import was requested, a full-import was run instead. This is no longer
the case. In Solr 1.4 delta import is run with last_index_time as the epoch
date (January 1, 1970, 00:00:00 GMT) if last_index_time is not available.

Versions of Major Components

----------------------------
Apache Lucene 2.9.1 (r832363 on 2.9 branch)
Apache Tika 0.4
Carrot2 3.1.0

Lucene Information
----------------

Since Solr is built on top of Lucene, many people add customizations to Solr
that are dependent on Lucene. Please see http://lucene.apache.org/java/2_9_0/,
especially http://lucene.apache.org/java/2_9_0/changes/Changes.html for more
information on the version of Lucene used in Solr.

Detailed Change List

----------------------

New Features
----------------------
 1. SOLR-560: Use SLF4J logging API rather then JDK logging. The packaged .war
file is
shipped with a JDK logging implementation, so logging configuration for
the .war should
be identical to solr 1.3. However, if you are using the .jar file, you can
select
which logging implementation to use by dropping a different binding.
See: http://www.slf4j.org/ (ryan)

2. SOLR-617: Allow configurable index deletion policy and provide a default

implementation which
allows deletion of commit points on various criteria such as number of commits,
age of commit
point and optimized status.
See
http://lucene.apache.org/java/2_3_2/api/org/apache/lucene/index/IndexDeletionPolicy
.html
(yonik, Noble Paul, Akshay Ukey via shalin)

3. SOLR-658: Allow Solr to load index from arbitrary directory in dataDir

(Noble Paul, Akshay Ukey via shalin)

4. SOLR-793: Add 'commitWithin' argument to the update add command. This behaves
similar to the global autoCommit maxTime argument except that it is set for
each request. (ryan)

5. SOLR-670: Add support for rollbacks in UpdateHandler. This allows user to

rollback all changes
since the last commit. (Noble Paul, koji via shalin)

6. SOLR-813: Adding DoubleMetaphone Filter and Factory. Similar to the

PhoneticFilter,
but this uses DoubleMetaphone specific calls (including alternate encoding)
(Todd Feak via ryan)

7. SOLR-680: Add StatsComponent. This gets simple statistics on matched numeric

fields,
including: min, max, mean, median, stddev. (koji, ryan)

- SOLR-1380: Added support for multi-valued fields (Harish Agarwal via

gsingers)

8. SOLR-561: Added Replication implemented in Java as a request handler. Supports

index replication
as well as configuration replication and exposes detailed statistics and
progress information
on the Admin page. Works on all platforms. (Noble Paul, yonik, Akshay Ukey,
shalin)

9. SOLR-746: Added "omitHeader" request parameter to omit the header from the
response.
(Noble Paul via shalin)

10. SOLR-651: Added TermVectorComponent for serving up term vector information,

plus IDF.
See http://wiki.apache.org/solr/TermVectorComponent (gsingers, Vaijanath N.
Rao, Noble Paul)

12. SOLR-795: SpellCheckComponent supports building indices on optimize if

configured in solrconfig.xml
(Jason Rennie, shalin)

13. SOLR-667: A LRU cache implementation based upon ConcurrentHashMap and other
techniques to reduce
contention and synchronization overhead, to utilize multiple CPU cores more
effectively.
(Fuad Efendi, Noble Paul, yonik via shalin)

14. SOLR-465: Add configurable DirectoryProvider so that alternate Directory

implementations can be specified via solrconfig.xml. The default
DirectoryProvider will use NIOFSDirectory for better concurrency
on non Windows platforms. (Mark Miller, TJ Laurenzo via yonik)

15. SOLR-822: Add CharFilter so that characters can be filtered (e.g. character
normalization)
before Tokenizer/TokenFilters. (koji)

16. SOLR-829: Allow slaves to request compressed files from master during
replication
(Simon Collins, Noble Paul, Akshay Ukey via shalin)

17. SOLR-877: Added TermsComponent for accessing Lucene's TermEnum capabilities.

Useful for auto suggest and possibly distributed search. Not distributed
search compliant. (gsingers)
- Added mincount and maxcount options (Khee Chin via gsingers)

18. SOLR-538: Add maxChars attribute for copyField function so that the length
limit for destination
can be specified.
(Georgios Stamatis, Lars Kotthoff, Chris Harris via koji)

19. SOLR-284: Added support for extracting content from binary documents like MS
Word and PDF using Apache Tika. See also contrib/extraction/CHANGES.txt (Eric
Pugh, Chris Harris, yonik, gsingers)

20. SOLR-819: Added factories for Arabic support (gsingers)

21. SOLR-781: Distributed search ability to sort field.facet values

lexicographically. facet.sort values "true" and "false" are
also deprecated and replaced with "count" and "lex".
(Lars Kotthoff via yonik)

22. SOLR-821: Add support for replication to copy conf file to slave with a
different name. This allows replication
of solrconfig.xml
(Noble Paul, Akshay Ukey via shalin)

23. SOLR-911: Add support for multi-select faceting by allowing filters to be

tagged and facet commands to exclude certain filters. This patch also
added the ability to change the output key for facets in the response, and
optimized distributed faceting refinement by lowering parsing overhead and
by making requests and responses smaller.

24. SOLR-876: WordDelimiterFilter now supports a splitOnNumerics

option, as well as a list of protected terms.
(Dan Rosher via hossman)

25. SOLR-928: SolrDocument and SolrInputDocument now implement the Map<String,?>

 interface. This should make plugging into other standard tools easier. (ryan)

26. SOLR-847: Enhance the snappull command in ReplicationHandler to accept

masterUrl.
(Noble Paul, Preetam Rao via shalin)

27. SOLR-540: Add support for globbing in field names to highlight.

For example, hl.fl=*_text will highlight all fieldnames ending with
_text. (Lars Kotthoff via yonik)

28. SOLR-906: Adding a StreamingUpdateSolrServer that writes update commands to

an open HTTP connection. If you are using solrj for bulk update requests
you should consider switching to this implementaion. However, note that
the error handling is not immediate as it is with the standard SolrServer.
(ryan)

29. SOLR-865: Adding support for document updates in binary format and
corresponding support in Solrj client.
(Noble Paul via shalin)

30. SOLR-763: Add support for Lucene's PositionFilter (Mck SembWever via shalin)

31. SOLR-966: Enhance the map() function query to take in an optional default value
(Noble Paul, shalin)

32. SOLR-820: Support replication on startup of master with new index. (Noble Paul,
Akshay Ukey via shalin)

33. SOLR-943: Make it possible to specify dataDir in solr.xml and accept the
dataDir as a request parameter for
the CoreAdmin create command. (Noble Paul via shalin)

34. SOLR-850: Addition of timeouts for distributed searching. Configurable through

'shard-socket-timeout' and
'shard-connection-timeout' parameters in SearchHandler. (Patrick O'Leary via
shalin)

35. SOLR-799: Add support for hash based exact/near duplicate document
handling. (Mark Miller, yonik)

36. SOLR-1026: Add protected words support to SnowballPorterFilterFactory

(ehatcher)

37. SOLR-739: Add support for OmitTf (Mark Miller via yonik)

38. SOLR-1046: Nested query support for the function query parser
and lucene query parser (the latter existed as an undocumented
feature in 1.3) (yonik)

39. SOLR-940: Add support for Lucene's Trie Range Queries by providing new
FieldTypes in
schema for int, float, long, double and date. Single-valued Trie based
fields with a precisionStep will index multiple precisions and enable
faster range queries. (Uwe Schindler, yonik, shalin)

40. SOLR-1038: Enhance CommonsHttpSolrServer to add docs in batch using an iterator

API (Noble Paul via shalin)

41. SOLR-844: A SolrServer implementation to front-end multiple solr servers and

provides load balancing and failover
support (Noble Paul, Mark Miller, hossman via shalin)

42. SOLR-939: ValueSourceRangeFilter/Query - filter based on values in a FieldCache

entry or on any arbitrary function of field values. (yonik)

43. SOLR-1095: Fixed performance problem in the StopFilterFactory and simplified

code. Added tests as well. (gsingers)

44. SOLR-1096: Introduced httpConnTimeout and httpReadTimeout in replication slave

configuration to avoid stalled
replication. (Jeff Newburn, Noble Paul, shalin)

45. SOLR-1115: <bool>on</bool> and <bool>yes</bool> work as expected in

solrconfig.xml. (koji)

46. SOLR-1099: A FieldAnalysisRequestHandler which provides the analysis

functionality of the web admin page as
a service. The AnalysisRequestHandler is renamed to
DocumentAnalysisRequestHandler which is enhanced with
query analysis and showMatch support. AnalysisRequestHandler is now deprecated.
Support for both
FieldAnalysisRequestHandler and DocumentAnalysisRequestHandler is also provided
in the Solrj client.
(Uri Boness, shalin)

47. SOLR-1106: Made CoreAdminHandler Actions pluggable so that additional actions

may be plugged in or the existing
ones can be overridden if needed. (Kay Kay, Noble Paul, shalin)

48. SOLR-1124: Add a top() function query that causes its argument to
have its values derived from the top level IndexReader, even when
invoked from a sub-reader. top() is implicitly used for the
ord() and rord() functions. (yonik)

49. SOLR-1110: Support sorting on trie fields with Distributed Search. (Mark
Miller, Uwe Schindler via shalin)

50. SOLR-1121: CoreAdminhandler should not need a core . This makes it possible to
start a Solr server w/o a core .(noble)

51. SOLR-769: Added support for clustering in contrib/clustering. See

http://wiki.apache.org/solr/ClusteringComponent for more info. (gsingers,
Stanislaw Osinski)

52. SOLR-1175: disable/enable replication on master side. added two commands

'enableReplication' and 'disableReplication' (noble)

53. SOLR-1179: DocSets can now be used as Lucene Filters via

DocSet.getTopFilter() (yonik)

54. SOLR-1116: Add a Binary FieldType (noble)

55. SOLR-1051: Support the merge of multiple indexes as a CoreAdmin and an update
command (Ning Li via shalin)

56. SOLR-1152: Snapshoot on ReplicationHandler should accept location as a request

parameter (shalin)
57. SOLR-1204: Enhance SpellingQueryConverter to handle UTF-8 instead of ASCII
only.
Use the NMTOKEN syntax for matching field names.
(Michael Ludwig, shalin)

58. SOLR-1189: Support providing username and password for basic HTTP
authentication in Java replication
(Matthew Gregg, shalin)

59. SOLR-243: Add configurable IndexReaderFactory so that alternate IndexReader

implementations
can be specified via solrconfig.xml. Note that using a custom IndexReader may
be incompatible
with ReplicationHandler (see comments in SOLR-1366). This should be treated as
an experimental feature.
(Andrzej Bialecki, hossman, Mark Miller, John Wang)

60. SOLR-1214: differentiate between solr home and instanceDir .deprecates the
method SolrResourceLoader#locateInstanceDir()
and it is renamed to locateSolrHome (noble)

61. SOLR-1216 : disambiguate the replication command names. 'snappull' becomes

'fetchindex' 'abortsnappull' becomes 'abortfetch' (noble)

62. SOLR-1145: Add capability to specify an infoStream log file for the underlying
Lucene IndexWriter in solrconfig.xml.
This is an advanced debug log file that can be used to aid developers in fixing
IndexWriter bugs. See the commented
out example in the example solrconfig.xml under the indexDefaults section.
(Chris Harris, Mark Miller)

63. SOLR-1256: Show the output of CharFilters in analysis.jsp. (koji)

64. SOLR-1266: Added stemEnglishPossessive option (default=true) to

WordDelimiterFilter
that allows disabling of english possessive stemming (removal of trailing 's
from tokens)
(Robert Muir via yonik)

65. SOLR-1237: firstSearcher and newSearcher can now be identified via the
CommonParams.EVENT (evt) parameter
in a request. This allows a RequestHandler or SearchComponent to know when a
newSearcher or firstSearcher
event happened. QuerySenderListender is the only implementation in Solr that
implements this, but outside
implementations may wish to. See the AbstractSolrEventListener for a helper
method. (gsingers)

66. SOLR-1343: Added HTMLStripCharFilter and marked HTMLStripReader,

HTMLStripWhitespaceTokenizerFactory and
HTMLStripStandardTokenizerFactory deprecated. To strip HTML tags,
HTMLStripCharFilter can be used
with an arbitrary Tokenizer. (koji)

67. SOLR-1275: Add expungeDeletes to DirectUpdateHandler2 (noble)

68. SOLR-1372: Enhance FieldAnalysisRequestHandler to accept field value from

content stream (ehatcher)
69. SOLR-1370: Show the output of CharFilters in FieldAnalysisRequestHandler (koji)

70. SOLR-1373: Add Filter query to admin/form.jsp

(Jason Rutherglen via hossman)

71. SOLR-1368: Add ms() function query for getting milliseconds from dates and for
high precision date subtraction, add sub() for subtracting other arguments.
(yonik)

72. SOLR-1156: Sort TermsComponent results by frequency (Matt Weber via yonik)

73. SOLR-1335 : load core properties from a properties file (noble)

74. SOLR-1385 : Add an 'enable' attribute to all plugins (noble)

75. SOLR-1414 : implicit core properties are not set for single core (noble)

76. SOLR-659 : Adds shards.start and shards.rows to distributed search

to allow more efficient bulk queries (those that retrieve many or all
documents). (Brian Whitman via yonik)

77. SOLR-1321: Add better support for efficient wildcard handling (Andrzej
Bialecki, Robert Muir, gsingers)

78. SOLR-1326 : New interface PluginInfoInitialized for all types of plugin (noble)

79. SOLR-1447 : Simple property injection. <mergePolicy> & <mergeScheduler>

syntaxes are now deprecated
(Jason Rutherglen, noble)

80. SOLR-908 : CommonGramsFilterFactory/CommonGramsQueryFilterFactory for

speeding up phrase queries containing common words by indexing
n-grams and using them at query time.
(Tom Burton-West, Jason Rutherglen via yonik)

81. SOLR-1292: Add FieldCache introspection to stats.jsp and JMX Monitoring via
a new SolrFieldCacheMBean. (hossman)

82. SOLR-1167: Solr Config now supports XInclude for XML engines that can support
it. (Bryan Talbot via gsingers)

83. SOLR-1478: Enable sort by Lucene docid. (ehatcher)

84. SOLR-1449: Add <lib> elements to solrconfig.xml to specifying additional

classpath directories and regular expressions. (hossman via yonik)

85. SOLR-1128: Added metadata output to extraction request handler "extract

only" option. (gsingers)

86. SOLR-1274: Added text serialization output for extractOnly

(Peter Wolanin, gsingers)

87. SOLR-768: DIH: Set last_index_time variable in full-import command.

(Wojtek Piaseczny, Noble Paul via shalin)

88. SOLR-811: Allow a "deltaImportQuery" attribute in SqlEntityProcessor

which is used for delta imports instead of DataImportHandler manipulating
the SQL itself. (Noble Paul via shalin)
89. SOLR-842: Better error handling in DataImportHandler with options to
abort, skip and continue imports. (Noble Paul, shalin)

90. SOLR-833: DIH: A DataSource to read data from a field as a reader. This
can be used, for example, to read XMLs residing as CLOBs or BLOBs in
databases. (Noble Paul via shalin)

91. SOLR-887: A DIH Transformer to strip HTML tags. (Ahmed Hammad via shalin)

92. SOLR-886: DataImportHandler should rollback when an import fails or it is

aborted (shalin)

93. SOLR-891: A DIH Transformer to read strings from Clob type.

(Noble Paul via shalin)

94. SOLR-812: Configurable JDBC settings in JdbcDataSource including optimized

defaults for read only mode. (David Smiley, Glen Newton, shalin)

95. SOLR-910: Add a few utility commands to the DIH admin page such as full
import, delta import, status, reload config. (Ahmed Hammad via shalin)

96. SOLR-938: Add event listener API for DIH import start and end.
(Kay Kay, Noble Paul via shalin)

97. SOLR-801: DIH: Add support for configurable pre-import and post-import
delete query per root-entity. (Noble Paul via shalin)

98. SOLR-988: Add a new scope for session data stored in Context to store
objects across imports. (Noble Paul via shalin)

99. SOLR-980: A PlainTextEntityProcessor which can read from any

DataSource<Reader> and output a String.
(Nathan Adams, Noble Paul via shalin)

100.SOLR-1003: XPathEntityprocessor must allow slurping all text from a given

xml node and its children. (Noble Paul via shalin)

101.SOLR-1001: Allow variables in various attributes of RegexTransformer,

HTMLStripTransformer and NumberFormatTransformer.
(Fergus McMenemie, Noble Paul, shalin)

102.SOLR-989: DIH: Expose running statistics from the Context API.

(Noble Paul, shalin)

103.SOLR-996: DIH: Expose Context to Evaluators. (Noble Paul, shalin)

104.SOLR-783: DIH: Enhance delta-imports by maintaining separate

last_index_time for each entity. (Jon Baer, Noble Paul via shalin)

105.SOLR-1033: Current entity's namespace is made available to all DIH

Transformers. This allows one to use an output field of TemplateTransformer
in other transformers, among other things.
(Fergus McMenemie, Noble Paul via shalin)

106.SOLR-1066: New methods in DIH Context to expose Script details.

ScriptTransformer changed to read scripts through the new API methods.
(Noble Paul via shalin)

107.SOLR-1062: A DIH LogTransformer which can log data in a given template

 format. (Jon Baer, Noble Paul via shalin)

108.SOLR-1065: A DIH ContentStreamDataSource which can accept HTTP POST data

in a content stream. This can be used to push data to Solr instead of
just pulling it from DB/Files/URLs. (Noble Paul via shalin)

109.SOLR-1061: Improve DIH RegexTransformer to create multiple columns from

regex groups. (Noble Paul via shalin)

110.SOLR-1059: Special DIH flags introduced for deleting documents by query or

id, skipping rows and stopping further transforms. Use $deleteDocById,
$deleteDocByQuery for deleting by id and query respectively. Use $skipRow
to skip the current row but continue with the document. Use $stopTransform
to stop further transformers. New methods are introduced in Context for
deleting by id and query. (Noble Paul, Fergus McMenemie, shalin)

111.SOLR-1076: JdbcDataSource should resolve DIH variables in all its

configuration parameters. (shalin)

112.SOLR-1055: Make DIH JdbcDataSource easily extensible by making the

createConnectionFactory method protected and return a
Callable<Connection> object. (Noble Paul, shalin)

113.SOLR-1058: DIH: JdbcDataSource can lookup javax.sql.DataSource using JNDI.

Use a jndiName attribute to specify the location of the data source.
(Jason Shepherd, Noble Paul via shalin)

114.SOLR-1083: A DIH Evaluator for escaping query characters.

(Noble Paul, shalin)

115.SOLR-934: A MailEntityProcessor to enable indexing mails from

POP/IMAP sources into a solr index. (Preetam Rao, shalin)

116.SOLR-1060: A DIH LineEntityProcessor which can stream lines of text from a

given file to be indexed directly or for processing with transformers and
child entities.
(Fergus McMenemie, Noble Paul, shalin)

117.SOLR-1127: Add support for DIH field name to be templatized.

(Noble Paul, shalin)

118.SOLR-1092: Added a new DIH command named 'import' which does not
automatically clean the index. This is useful and more appropriate when one
needs to import only some of the entities.
(Noble Paul via shalin)

119.SOLR-1153: DIH 'deltaImportQuery' is honored on child entities as well

(noble)

120.SOLR-1230: Enhanced dataimport.jsp to work with all DataImportHandler

request handler configurations, rather than just a hardcoded /dataimport
handler. (ehatcher)

121.SOLR-1235: disallow period (.) in DIH entity names (noble)

122.SOLR-1234: Multiple DIH does not work because all of them write to
dataimport.properties. Use the handler name as the properties file name
(noble)
123.SOLR-1348: Support binary field type in convertType logic in DIH
JdbcDataSource (shalin)

124.SOLR-1406: DIH: Make FileDataSource and FileListEntityProcessor to be more

extensible (Luke Forehand, shalin)

125.SOLR-1437: DIH: XPathEntityProcessor can deal with xpath syntaxes such as

//tagname , /root//tagname (Fergus McMenemie via noble)

Optimizations
----------------------
1. SOLR-374: Use IndexReader.reopen to save resources by re-using parts of the
index that haven't changed. (Mark Miller via yonik)

2. SOLR-808: Write string keys in Maps as extern strings in the javabin format.
(Noble Paul via shalin)

3. SOLR-475: New faceting method with better performance and smaller memory usage
for
multi-valued fields with many unique values but relatively few values per
document.
Controllable via the facet.method parameter - "fc" is the new default method
and "enum"
is the original method. (yonik)

4. SOLR-970: Use an ArrayList in SolrPluginUtils.parseQueryStrings

since we know exactly how long the List will be in advance.
(Kay Kay via hossman)

5. SOLR-1002: Change SolrIndexSearcher to use insertWithOverflow

with reusable priority queue entries to reduce the amount of
generated garbage during searching. (Mark Miller via yonik)

6. SOLR-971: Replace StringBuffer with StringBuilder for instances that do not

require thread-safety.
(Kay Kay via shalin)

7. SOLR-921: SolrResourceLoader must cache short class name vs fully qualified

classname
(Noble Paul, hossman via shalin)

8. SOLR-973: CommonsHttpSolrServer writes the xml directly to the server.

(Noble Paul via shalin)

9. SOLR-1108: Remove un-needed synchronization in SolrCore constructor.

(Noble Paul via shalin)

10. SOLR-1166: Speed up docset/filter generation by avoiding top-level

score() call and iterating over leaf readers with TermDocs. (yonik)

11. SOLR-1169: SortedIntDocSet - a new small set implementation

that saves memory over HashDocSet, is faster to construct,
is ordered for easier implementation of skipTo, and is faster
in the general case. (yonik)

12. SOLR-1165: Use Lucene Filters and pass them down to the Lucene
search methods to filter earlier and improve performance. (yonik)
13. SOLR-1111: Use per-segment sorting to share fieldcache elements
across unchanged segments. This saves memory and reduces
commit times for incremental updates to the index. (yonik)

14. SOLR-1188: Minor efficiency improvement in TermVectorComponent related to

ignoring positions or offsets (gsingers)

15. SOLR-1150: Load Documents for Highlighting one at a time rather than
all at once to avoid OOM with many large Documents. (Siddharth Gargate via Mark
Miller)

16. SOLR-1353: Implement and use reusable token streams for analysis. (Robert Muir,
yonik)

17. SOLR-1296: Enables setting IndexReader's termInfosIndexDivisor via a new

attribute to StandardIndexReaderFactory. Enables
setting termIndexInterval to IndexWriter via SolrIndexConfig. (Jason
Rutherglen, hossman, gsingers)

18. SOLR-846: DIH: Reduce memory consumption during delta import by removing
keys when used (Ricky Leung, Noble Paul via shalin)

19. SOLR-974: DataImportHandler skips commit if no data has been updated.

(Wojtek Piaseczny, shalin)

20. SOLR-1004: DIH: Check for abort more frequently during delta-imports.
(Marc Sturlese, shalin)

21. SOLR-1098: DIH DateFormatTransformer can cache the format objects.

(Noble Paul via shalin)

22. SOLR-1465: Replaced string concatenations with StringBuilder append

calls in DIH XPathRecordReader. (Mark Miller, shalin)

Bug Fixes
----------------------
1. SOLR-774: Fixed logging level display (Sean Timm via Otis Gospodnetic)

2. SOLR-771: CoreAdminHandler STATUS should display 'normalized' paths (koji,

hossman, shalin)

3. SOLR-532: WordDelimiterFilter now respects payloads and other attributes of the

original Token by
using Token.clone() (Tricia Williams, gsingers)

4. SOLR-805: DisMax queries are not being cached in QueryResultCache (Todd Feak
via koji)

5. SOLR-751: WordDelimiterFilter didn't adjust the start offset of single

tokens that started with delimiters, leading to incorrect highlighting.
(Stefan Oestreicher via yonik)

7. SOLR-843: SynonymFilterFactory cannot handle multiple synonym files correctly

(koji)

8. SOLR-840: BinaryResponseWriter does not handle incompatible data in fields

(Noble Paul via shalin)

9. SOLR-803: CoreAdminRequest.createCore fails because name parameter isn't set

(Sean Colombo via ryan)

10. SOLR-869: Fix file descriptor leak in SolrResourceLoader#getLines (Mark Miller,

shalin)

11. SOLR-872: Better error message for incorrect copyField destination (Noble Paul
via shalin)

12. SOLR-879: Enable position increments in the query parser and fix the
example schema to enable position increments for the stop filter in
both the index and query analyzers to fix the bug with phrase queries
with stopwords. (yonik)

13. SOLR-836: Add missing "a" to the example stopwords.txt (yonik)

14. SOLR-892: Fix serialization of booleans for PHPSerializedResponseWriter

(yonik)

15. SOLR-898: Fix null pointer exception for the JSON response writer
based formats when nl.json=arrarr with null keys. (yonik)

16. SOLR-901: FastOutputStream ignores write(byte[]) call. (Noble Paul via shalin)

17. SOLR-807: BinaryResponseWriter writes fieldType.toExternal if it is not a

supported type,
otherwise it writes fieldType.toObject. This fixes the bug with
encoding/decoding UUIDField.
(koji, Noble Paul, shalin)

18. SOLR-863: SolrCore.initIndex should close the directory it gets for clearing
the lock and
use the DirectoryFactory. (Mark Miller via shalin)

19. SOLR-802: Fix a potential null pointer error in the distributed FacetComponent
(David Bowen via ryan)

20. SOLR-346: Use perl regex to improve accuracy of finding latest snapshot in
snapinstaller (billa)

21. SOLR-830: Use perl regex to improve accuracy of finding latest snapshot in
snappuller (billa)

22. SOLR-897: Fixed Argument list too long error when there are lots of
snapshots/backups (Dan Rosher via billa)

23. SOLR-925: Fixed highlighting on fields with multiValued="true" and

termOffsets="true" (koji)

24. SOLR-902: FastInputStream#read(byte b[], int off, int len) gives incorrect
results when amount left to read is less
than buffer size (Noble Paul via shalin)

25. SOLR-978: Old files are not removed from slaves after replication (Jaco, Noble
Paul, shalin)

26. SOLR-883: Implicit properties are not set for Cores created through CoreAdmin
(Noble Paul via shalin)

27. SOLR-991: Better error message when parsing solrconfig.xml fails due to
malformed XML. Error message notes the name
of the file being parsed. (Michael Henson via shalin)

28. SOLR-1008: Fix stats.jsp XML encoding for <stat> item entries with ampersands
in their names. (ehatcher)

29. SOLR-976: deleteByQuery is ignored when deleteById is placed prior to

deleteByQuery in a <delete>.
Now both delete by id and delete by query can be specified at the same time as
follows.
<delete>
<id>05991</id><id>06000</id>
<query>office:Bridgewater</query><query>office:Osaka</query>
</delete>
(koji)

30. SOLR-1016: HTTP 503 error changes 500 in SolrCore (koji)

31. SOLR-1015: Incomplete information in replication admin page and http command
response when server
is both master and slave i.e. when server is a repeater (Akshay Ukey via
shalin)

32. SOLR-1018: Slave is unable to replicate when server acts as repeater (as both
master and slave)
(Akshay Ukey, Noble Paul via shalin)

33. SOLR-1031: Fix XSS vulnerability in schema.jsp (Paul Lovvik via ehatcher)

34. SOLR-1064: registry.jsp incorrectly displaying info for last core initialized
regardless of what the current core is. (hossman)

35. SOLR-1072: absolute paths used in sharedLib attribute were

incorrectly treated as relative paths. (hossman)

36. SOLR-1104: Fix some rounding errors in LukeRequestHandler's histogram (hossman)

37. SOLR-1125: Use query analyzer rather than index analyzer for queryFieldType in
QueryElevationComponent
(koji)

38. SOLR-1126: Replicated files have incorrect timestamp (Jian Han Guo, Jeff
Newburn, Noble Paul via shalin)

39. SOLR-1094: Incorrect value of correctlySpelled attribute in some cases (David

Smiley, Mark Miller via shalin)

40. SOLR-965: Better error message when <pingQuery> is not configured.

(Mark Miller via hossman)

41. SOLR-1135: Java replication creates Snapshot in the directory where Solr was
launched (Jianhan Guo via shalin)

42. SOLR-1138: Query Elevation Component now gracefully handles missing queries.
(gsingers)

43. SOLR-929: LukeRequestHandler should return "dynamicBase" only if the field is

dynamic.
(Peter Wolanin, koji)
44. SOLR-1141: NullPointerException during snapshoot command in java based
replication (Jian Han Guo, shalin)

45. SOLR-1078: Fixes to WordDelimiterFilter to avoid splitting or dropping

international non-letter characters such as non spacing marks. (yonik)

46. SOLR-825, SOLR-1221: Enables highlighting for range/wildcard/fuzzy/prefix

queries if using hl.usePhraseHighlighter=true
and hl.highlightMultiTerm=true. Also make both options default to true. (Mark
Miller, yonik)

47. SOLR-1174: Fix Logging admin form submit url for multicore. (Jacob Singh via
shalin)

48. SOLR-1182: Fix bug in OrdFieldSource#equals which could cause a bug with
OrdFieldSource caching
on OrdFieldSource#hashcode collisions. (Mark Miller)

49. SOLR-1207: equals method should compare this and other of DocList in DocSetBase
(koji)

50. SOLR-1242: Human readable JVM info from system handler does integer cutoff
rounding, even when dealing
with GB. Fixed to round to one decimal place. (Jay Hill, Mark
Miller)

51. SOLR-1243: Admin RequestHandlers should not be cached over HTTP. (Mark Miller)

52. SOLR-1260: Fix implementations of set operations for DocList subclasses

and fix a bug in HashDocSet construction when offset != 0. These bugs
never manifested in normal Solr use and only potentially affect
custom code. (yonik)

53. SOLR-1171: Fix LukeRequestHandler so it doesn't rely on SolrQueryParser

and report incorrect stats when field names contain characters
SolrQueryParser considers special.
(hossman)

54. SOLR-1317: Fix CapitalizationFilterFactory to work when keep parameter is not

specified.
(ehatcher)

55. SOLR-1342: CapitalizationFilterFactory uses incorrect term length calculations.

(Robert Muir via Mark Miller)

56. SOLR-1359: DoubleMetaphoneFilter didn't index original tokens if there was no

alternative, and could incorrectly skip or reorder tokens. (yonik)

57. SOLR-1360: Prevent PhoneticFilter from producing duplicate tokens. (yonik)

58. SOLR-1371: LukeRequestHandler/schema.jsp errored if schema had no

uniqueKey field. The new test for this also (hopefully) adds some
future proofing against similar bugs in the future. As a side
effect QueryElevationComponentTest was refactored, and a bug in
that test was found. (hossman)

59. SOLR-914: General finalize() improvements. No finalizer delegates

to the respective close/destroy method w/o first checking if it's
 already been closed/destroyed; if it hasn't a, SEVERE error is
logged first. (noble, hossman)

60. SOLR-1362: WordDelimiterFilter had inconsistent behavior when setting

the position increment of tokens following a token consisting of all
delimiters, and could additionally lose big position increments.
(Robert Muir, yonik)

61. SOLR-1091: Jetty's use of CESU-8 for code points outside the BMP
resulted in invalid output from the serialized PHP writer. (yonik)

62. SOLR-1103: LukeRequestHandler (and schema.jsp) have been fixed to

include the "1" (ie: 2**0) bucket in the term histogram data.
(hossman)

63. SOLR-1398: Add offset corrections in PatternTokenizerFactory.

(Anders Melchiorsen, koji)

64. SOLR-1400: Properly handle zero-length tokens in TrimFilter. This

was not a bug in any released version. (Peter Wolanin, gsingers)

65. SOLR-1071: spellcheck.extendedResults returns an invalid JSON response

when count > 1. To fix, the extendedResults format was changed.
(Uri Boness, yonik)

66. SOLR-1381: Fixed improper handling of fields that have only term positions and
not term offsets during Highlighting (Thorsten Fischer, gsingers)

67. SOLR-1427: Fixed registry.jsp issue with MBeans (gsingers)

68. SOLR-1468: SolrJ's XML response parsing threw an exception for null
names, such as those produced when facet.missing=true (yonik)

69. SOLR-1471: Fixed issue with calculating missing values for facets in single
valued cases in Stats Component.
This is not correctly calculated for the multivalued case. (James Miller,
gsingers)

70. SOLR-1481: Fixed omitHeader parameter for PHP ResponseWriter. (Jun Ohtani via
billa)

71. SOLR-1448: Add weblogic.xml to solr webapp to enable correct operation in

WebLogic. (Ilan Rabinovitch via yonik)

72. SOLR-1504: empty char mapping can cause ArrayIndexOutOfBoundsException in

analysis.jsp and co.
(koji)

73. SOLR-1394: HTMLStripCharFilter split tokens that contained entities and

often calculated offsets incorrectly for entities.
(Anders Melchiorsen via yonik)

74. SOLR-1517: Admin pages could stall waiting for localhost name resolution
if reverse DNS wasn't configured; this was changed so the DNS resolution
is attempted only once the first time an admin page is loaded.
(hossman)

75. SOLR-1529: More than 8 deleteByQuery commands in a single request

caused an error to be returned, although the deletes were
 still executed. (asmodean via yonik)

76. SOLR-800: Deep copy collections to avoid ConcurrentModificationException

in XPathEntityprocessor while streaming
(Kyle Morrison, Noble Paul via shalin)

77. SOLR-823: Request parameter variables ${dataimporter.request.xxx} are not

resolved in DIH (Mck SembWever, Noble Paul, shalin)

78. SOLR-728: Add synchronization to avoid race condition of multiple DIH

imports working concurrently (Walter Ferrara, shalin)

79. SOLR-742: Add ability to create dynamic fields with custom

DataImportHandler transformers (Wojtek Piaseczny, Noble Paul, shalin)

80. SOLR-832: Rows parameter is not honored in DIH non-debug mode and can
abort a running import in debug mode. (Akshay Ukey, shalin)

81. SOLR-838: The DIH VariableResolver obtained from a DataSource's context

does not have current data. (Noble Paul via shalin)

82. SOLR-864: DataImportHandler does not catch and log Errors (shalin)

83. SOLR-873: Fix case-sensitive field names and columns (Jon Baer, shalin)

84. SOLR-893: Unable to delete documents via SQL and deletedPkQuery with
deltaimport (Dan Rosher via shalin)

85. SOLR-888: DIH DateFormatTransformer cannot convert non-string type

(Amit Nithian via shalin)

86. SOLR-841: DataImportHandler should throw exception if a field does not

have column attribute (Michael Henson, shalin)

87. SOLR-884: CachedSqlEntityProcessor should check if the cache key is

present in the query results (Noble Paul via shalin)

88. SOLR-985: Fix thread-safety issue with DIH TemplateString for concurrent
imports with multiple cores. (Ryuuichi Kumai via shalin)

89. SOLR-999: DIH XPathRecordReader fails on XMLs with nodes mixed with
CDATA content. (Fergus McMenemie, Noble Paul via shalin)

90. SOLR-1000: DIH FileListEntityProcessor should not apply fileName filter to

directory names. (Fergus McMenemie via shalin)

91. SOLR-1009: Repeated column names result in duplicate values.

(Fergus McMenemie, Noble Paul via shalin)

92. SOLR-1017: Fix DIH thread-safety issue with last_index_time for concurrent
imports in multiple cores due to unsafe usage of SimpleDateFormat by
multiple threads. (Ryuuichi Kumai via shalin)

93. SOLR-1024: Calling abort on DataImportHandler import commits data instead

of calling rollback. (shalin)

94. SOLR-1037: DIH should not add null values in a row returned by
EntityProcessor to documents. (shalin)
95. SOLR-1040: DIH XPathEntityProcessor fails with an xpath like
/feed/entry/link[@type='text/html']/@href (Noble Paul via shalin)

96. SOLR-1042: Fix memory leak in DIH by making TemplateString non-static

member in VariableResolverImpl (Ryuuichi Kumai via shalin)

97. SOLR-1053: IndexOutOfBoundsException in DIH SolrWriter.getResourceAsString

when size of data-config.xml is a multiple of 1024 bytes.
(Herb Jiang via shalin)

98. SOLR-1077: IndexOutOfBoundsException with useSolrAddSchema in DIH

XPathEntityProcessor. (Sam Keen, Noble Paul via shalin)

99. SOLR-1080: DIH RegexTransformer should not replace if regex is not matched.
(Noble Paul, Fergus McMenemie via shalin)

100.SOLR-1090: DataImportHandler should load the data-config.xml using UTF-8

encoding. (Rui Pereira, shalin)

101.SOLR-1146: ConcurrentModificationException in DataImporter.getStatusMessages

(Walter Ferrara, Noble Paul via shalin)

102.SOLR-1229: Fixes for DIH deletedPkQuery, particularly when using

transformed Solr unique id's
(Lance Norskog, Noble Paul via ehatcher)

103.SOLR-1286: Fix the IH commit parameter always defaulting to "true" even

if "false" is explicitly passed in. (Jay Hill, Noble Paul via ehatcher)

104.SOLR-1323: Reset XPathEntityProcessor's $hasMore/$nextUrl when fetching

next URL (noble, ehatcher)

105.SOLR-1450: DIH: Jdbc connection properties such as batchSize are not

applied if the driver jar is placed in solr_home/lib.
(Steve Sun via shalin)

106.SOLR-1474: DIH Delta-import should run even if last_index_time is not set.

(shalin)

Other Changes
----------------------
1. Upgraded to Lucene 2.4.0 (yonik)

2. SOLR-805: Upgraded to Lucene 2.9-dev (r707499) (koji)

3. DumpRequestHandler (/debug/dump): changed 'fieldName' to 'sourceInfo'.

(ehatcher)

4. SOLR-852: Refactored common code in CSVRequestHandler and

XMLUpdateRequestHandler (gsingers, ehatcher)

5. SOLR-871: Removed dependency on stax-utils.jar. If you using solr.jar and

running
java 6, you can also remove woodstox and geronimo. (ryan)

6. SOLR-465: Upgraded to Lucene 2.9-dev (r719351) (shalin)

7. SOLR-889: Upgraded to commons-io-1.4.jar and commons-fileupload-1.2.1.jar

(ryan)

8. SOLR-875: Upgraded to Lucene 2.9-dev (r723985) and consolidated the BitSet

implementations (Michael Busch, gsingers)

9. SOLR-819: Upgraded to Lucene 2.9-dev (r724059) to get access to Arabic public

constructors (gsingers)

10. SOLR-900: Moved solrj into /src/solrj. The contents of solr-common.jar is now
included
in the solr-solrj.jar. (ryan)

11. SOLR-924: Code cleanup: make all existing finalize() methods call
super.finalize() in a finally block. All current instances extend
Object, so this doesn't fix any bugs, but helps protect against
future changes. (Kay Kay via hossman)

12. SOLR-885: NamedListCodec is renamed to JavaBinCodec and returns Object instead

of NamedList.
(Noble Paul, yonik via shalin)

13. SOLR-84: Use new Solr logo in admin (Michiel via koji)

14. SOLR-981: groupId for Woodstox dependency in maven solrj changed to

org.codehaus.woodstox (Tim Taranov via shalin)

15. Upgraded to Lucene 2.9-dev r738218 (yonik)

16. SOLR-959: Refactored TestReplicationHandler to remove hardcoded port numbers

(hossman, Akshay Ukey via shalin)

17. Upgraded to Lucene 2.9-dev r742220 (yonik)

18. SOLR-1022: Better "ignored" field in example schema.xml (Peter Wolanin via
hossman)

19. SOLR-967: New type-safe constructor for NamedList (Kay Kay via hossman)

20. SOLR-1036: Change default QParser from "lucenePlusSort" to "lucene" to

reduce confusion of semicolon splitting behavior when no sort param is
specified (hossman)

21. Upgraded to Lucene 2.9-dev r752164 (shalin)

22. SOLR-1068: Use fsync on replicated index and configuration files (yonik, Noble
Paul, shalin)

23. SOLR-952: Cleanup duplicated code in deprecated HighlightingUtils (hossman)

24. Upgraded to Lucene 2.9-dev r764281 (shalin)

25. SOLR-1079: Rename omitTf to omitTermFreqAndPositions (shalin)

26. SOLR-804: Added Lucene's misc contrib JAR (rev 764281). (gsingers)

27. Upgraded to Lucene 2.9-dev r768228 (shalin)

28. Upgraded to Lucene 2.9-dev r768336 (shalin)

29. SOLR-997: Wait for a longer time for slave to complete replication in
TestReplicationHandler
(Mark Miller via shalin)

30. SOLR-748: FacetComponent helper classes are made public as an experimental API.
(Wojtek Piaseczny via shalin)

31. Upgraded to Lucene 2.9-dev 773862 (Mark Miller)

32. Upgraded to Lucene 2.9-dev r776177 (shalin)

33. SOLR-1149: Made QParserPlugin and related classes extendible as an experimental

API.
(Kaktu Chakarabati via shalin)

34. Upgraded to Lucene 2.9-dev r779312 (yonik)

35. SOLR-786: Refactor DisMaxQParser to allow overriding certain features of

DisMaxQParser
(Wojciech Biela via shalin)

36. SOLR-458: Add equals and hashCode methods to NamedList (Stefan Rinner, shalin)

37. SOLR-1184: Add option in solrconfig to open a new IndexReader rather than
using reopen. Done mainly as a fail-safe in the case that a user runs into
a reopen bug/issue. (Mark Miller)

38. SOLR-1215 use double quotes to enclose attributes in solr.xml (noble)

39. SOLR-1151: add dynamic copy field and maxChars example to example schema.xml.
(Peter Wolanin, Mark Miller)

40. SOLR-1233: remove /select?qt=/whatever restriction on /-prefixed request

handlers.
(ehatcher)

41. SOLR-1257: logging.jsp has been removed and now passes through to the
hierarchical log level tool added in Solr 1.3. Users still
hitting "/admin/logging.jsp" should switch to "/admin/logging".
(hossman)

42. Upgraded to Lucene 2.9-dev r794238. Other changes include:

- LUCENE-1614 - Use Lucene's DocIdSetIterator.NO_MORE_DOCS as the sentinel
value.
- LUCENE-1630 - Add acceptsDocsOutOfOrder method to Collector implementations.
- LUCENE-1673, LUCENE-1701 - Trie has moved to Lucene core and renamed to
NumericRangeQuery.
- LUCENE-1662, LUCENE-1687 - Replace usage of ExtendedFieldCache by FieldCache.
(shalin)

42. SOLR-1241: Solr's CharFilter has been moved to Lucene. Remove CharFilter and
related classes
from Solr and use Lucene's corresponding code (koji via shalin)

43. SOLR-1261: Lucene trunk renamed RangeQuery & Co to TermRangeQuery (Uwe

Schindler via shalin)

44. Upgraded to Lucene 2.9-dev r801856 (Mark Miller)

45. SOLR-1276: Added StatsComponentTest (Rafał Kuć, gsingers)

46. SOLR-1377: The TokenizerFactory API has changed to explicitly return a

Tokenizer
rather then a TokenStream (that may be or may not be a Tokenizer). This change
is required to take advantage of the Token reuse improvements in lucene 2.9.
(ryan)

47. SOLR-1410: Log a warning if the deprecated charset option is used

on GreekLowerCaseFilterFactory, RussianStemFilterFactory,
RussianLowerCaseFilterFactory or RussianLetterTokenizerFactory.
(Robert Muir via hossman)

48. SOLR-1423: Due to LUCENE-1906, Solr's tokenizer should use

Tokenizer.correctOffset() instead of CharStream.correctOffset().
(Uwe Schindler via koji)

49. SOLR-1319, SOLR-1345: Upgrade Solr Highlighter classes to new Lucene

Highlighter API. This upgrade has
resulted in a back compat break in the DefaultSolrHighlighter class -
getQueryScorer is no longer
protected. If you happened to be overriding that method in custom code, overide
getHighlighter instead.
Also, HighlightingUtils#getQueryScorer has been removed as it was deprecated
and backcompat has been
broken with it anyway. (Mark Miller)

50. SOLR-1357 SolrInputDocument cannot process dynamic fields (Lars Grote via
noble)

51. SOLR-1075: Upgrade to Tika 0.3. See

http://www.apache.org/dist/lucene/tika/CHANGES-0.3.txt (gsingers)

52. SOLR-1310: Upgrade to Tika 0.4. Note there are some differences in
detecting Languages now in extracting request handler.
See
http://www.lucidimagination.com/search/document/d6f1899a85b2a45c/vote_apache_tika_0
_4_release_candidate_2#d6f1899a85b2a45c
for discussion on language detection.
See http://www.apache.org/dist/lucene/tika/CHANGES-0.4.txt. (gsingers)

53. SOLR-782: DIH: Refactored SolrWriter to make it a concrete class and

removed wrappers over SolrInputDocument. Refactored to load Evaluators
lazily. Removed multiple document nodes in the configuration xml. Removed
support for 'default' variables, they are automatically available as
request parameters. (Noble Paul via shalin)

54. SOLR-964: DIH: XPathEntityProcessor now ignores DTD validations

(Fergus McMenemie, Noble Paul via shalin)

55. SOLR-1029: DIH: Standardize Evaluator parameter parsing and added helper
functions for parsing all evaluator parameters in a standard way.
(Noble Paul, shalin)

56. SOLR-1081: Change DIH EventListener to be an interface so that components

such as an EntityProcessor or a Transformer can act as an event listener.
(Noble Paul, shalin)

57. SOLR-1027: DIH: Alias the 'dataimporter' namespace to a shorter name 'dih'.
 (Noble Paul via shalin)

58. SOLR-1084: Better error reporting when DIH entity name is a reserved word
and data-config.xml root node is not <dataConfig>.
(Noble Paul via shalin)

59. SOLR-1087: Deprecate 'where' attribute in CachedSqlEntityProcessor in

favor of cacheKey and cacheLookup. (Noble Paul via shalin)

60. SOLR-969: Change the FULL_DUMP, DELTA_DUMP, FIND_DELTA constants in DIH

Context to String. Change Context.currentProcess() to return a string
instead of an integer. (Kay Kay, Noble Paul, shalin)

61. SOLR-1120: Simplified DIH EntityProcessor API by moving logic for applying
transformers and handling multi-row outputs from Transformers into an
EntityProcessorWrapper class. The behavior of the method
EntityProcessor#destroy has been modified to be called once per parent-row
at the end of row. A new method EntityProcessor#close is added which is
called at the end of import. A new method
Context#getResolvedEntityAttribute is added which returns the resolved
value of an entity's attribute. Introduced a DocWrapper which takes care
of maintaining document level session variables.
(Noble Paul, shalin)

62. SOLR-1265: Add DIH variable resolving for URLDataSource properties like
baseUrl. (Chris Eldredge via ehatcher)

63. SOLR-1269: Better error messages from DIH JdbcDataSource when JDBC Driver
name or SQL is incorrect. (ehatcher, shalin)

Build
----------------------
1. SOLR-776: Added in ability to sign artifacts via Ant for releases (gsingers)

2. SOLR-854: Added run-example target (Mark Miller via ehatcher)

3. SOLR-1054:Fix dist-src target for DataImportHandler (Ryuuichi Kumai via shalin)

4. SOLR-1219: Added proxy.setup target (koji)

5. SOLR-1386: In build.xml, use longfile="gnu" in tar task to avoid warnings about

long file names
(Mark Miller via shalin)

6. SOLR-1441: Make it possible to run all tests in a package (shalin)

Documentation
----------------------
1. SOLR-789: The javadoc of RandomSortField is not readable (Nicolas Lalevée via
koji)

2. SOLR-962: Note about null handling in ModifiableSolrParams.add javadoc

(Kay Kay via hossman)

3. SOLR-1409: Added Solr Powered By Logos

4. SOLR-1369: Add HSQLDB Jar to example-DIH, unzip database and update

 instructions.

================== Release 1.3.0 ==================

Upgrading from Solr 1.2

-----------------------
IMPORTANT UPGRADE NOTE: In a master/slave configuration, all searchers/slaves
should be upgraded before the master! If the master were to be updated
first, the older searchers would not be able to read the new index format.

The Porter snowball based stemmers in Lucene were updated (LUCENE-1142),

and are not guaranteed to be backward compatible at the index level
(the stem of certain words may have changed). Re-indexing is recommended.

Older Apache Solr installations can be upgraded by replacing

the relevant war file with the new version. No changes to configuration
files should be needed.

This version of Solr contains a new version of Lucene implementing

an updated index format. This version of Solr/Lucene can still read
and update indexes in the older formats, and will convert them to the new
format on the first index change. Be sure to backup your index before
upgrading in case you need to downgrade.

Solr now recognizes HTTP Request headers related to HTTP Caching (see
RFC 2616 sec13) and will by default respond with "304 Not Modified"
when appropriate. This should only affect users who access Solr via
an HTTP Cache, or via a Web-browser that has an internal cache, but if
you wish to suppress this behavior an '<httpCaching never304="true"/>'
option can be added to your solrconfig.xml. See the wiki (or the
example solrconfig.xml) for more details...
http://wiki.apache.org/solr/SolrConfigXml#HTTPCaching

In Solr 1.2, DateField did not enforce the canonical representation of

the ISO 8601 format when parsing incoming data, and did not generation
the canonical format when generating dates from "Date Math" strings
(particularly as it pertains to milliseconds ending in trailing zeros).
As a result equivalent dates could not always be compared properly.
This problem is corrected in Solr 1.3, but DateField users that might
have been affected by indexing inconsistent formats of equivalent
dates (ie: 1995-12-31T23:59:59Z vs 1995-12-31T23:59:59.000Z) may want
to consider reindexing to correct these inconsistencies. Users who
depend on some of the the "broken" behavior of DateField in Solr 1.2
(specificly: accepting any input that ends in a 'Z') should consider
using the LegacyDateField class as a possible alternative. Users that
desire 100% backwards compatibility should consider using the Solr 1.2
version of DateField.

Due to some changes in the lifecycle of TokenFilterFactories, users of

Solr 1.2 who have written Java code which constructs new instances of
StopFilterFactory, SynonymFilterFactory, or EnglishProterFilterFactory
will need to modify their code by adding a line like the following
prior to using the factory object...
factory.inform(SolrCore.getSolrCore().getSolrConfig().getResourceLoader());
These lifecycle changes do not affect people who use Solr "out of the
box" or who have developed their own TokenFilterFactory plugins. More
info can be found in SOLR-594.
The python client that used to ship with Solr is no longer included in
the distribution (see client/python/README.txt).

Detailed Change List

--------------------

New Features
1. SOLR-69: Adding MoreLikeThisHandler to search for similar documents using
lucene contrib/queries MoreLikeThis. MoreLikeThis is also available from
the StandardRequestHandler using ?mlt=true. (bdelacretaz, ryan)

2. SOLR-253: Adding KeepWordFilter and KeepWordFilterFactory. A TokenFilter

that keeps tokens with text in the registered keeplist. This behaves like
the inverse of StopFilter. (ryan)

3. SOLR-257: WordDelimiterFilter has a new parameter splitOnCaseChange,

which can be set to 0 to disable splitting "PowerShot" => "Power" "Shot".
(klaas)

4. SOLR-193: Adding SolrDocument and SolrInputDocument to represent documents

outside of the lucene Document infrastructure. This class will be used
by clients and for processing documents. (ryan)

5. SOLR-244: Added ModifiableSolrParams - a SolrParams implementation that

help you change values after initialization. (ryan)

6. SOLR-20: Added a java client interface with two implementations. One

implementation uses commons httpclient to connect to solr via HTTP. The
other connects to solr directly. Check client/java/solrj. This addition
also includes tests that start jetty and test a connection using the full
HTTP request cycle. (Darren Erik Vengroff, Will Johnson, ryan)

7. SOLR-133: Added StaxUpdateRequestHandler that uses StAX for XML parsing.

This implementation has much better error checking and lets you configure
a custom UpdateRequestProcessor that can selectively process update
requests depending on the request attributes. This class will likely
replace XmlUpdateRequestHandler. (Thorsten Scherler, ryan)

8. SOLR-264: Added RandomSortField, a utility field with a random sort order.

The seed is based on a hash of the field name, so a dynamic field
of this type is useful for generating different random sequences.
This field type should only be used for sorting or as a value source
in a FunctionQuery (ryan, hossman, yonik)

9. SOLR-266: Adding show=schema to LukeRequestHandler to show the parsed

schema fields and field types. (ryan)

10. SOLR-133: The UpdateRequestHandler now accepts multiple delete options

within a single request. For example, sending:
<delete><id>1</id><id>2</id></delete> will delete both 1 and 2. (ryan)

11. SOLR-269: Added UpdateRequestProcessor plugin framework. This provides

a reasonable place to process documents after they are parsed and
before they are committed to the index. This is a good place for custom
document manipulation or document based authorization. (yonik, ryan)

12. SOLR-260: Converting to a standard PluginLoader framework. This reworks

RequestHandlers, FieldTypes, and QueryResponseWriters to share the same
base code for loading and initializing plugins. This adds a new
 configuration option to define the default RequestHandler and
QueryResponseWriter in XML using default="true". (ryan)

13. SOLR-225: Enable pluggable highlighting classes. Allow configurable

highlighting formatters and Fragmenters. (ryan)

14. SOLR-273/376/452/516: Added hl.maxAnalyzedChars highlighting parameter,

defaulting
to 50k, hl.alternateField, which allows the specification of a backup
field to use as summary if no keywords are matched, and hl.mergeContiguous,
which combines fragments if they are adjacent in the source document.
(klaas, Grant Ingersoll, Koji Sekiguchi via klaas)

15. SOLR-291: Control maximum number of documents to cache for any entry
in the queryResultCache via queryResultMaxDocsCached solrconfig.xml
entry. (Koji Sekiguchi via yonik)

16. SOLR-240: New <lockType> configuration setting in <mainIndex> and

<indexDefaults> blocks supports all Lucene builtin LockFactories.
'single' is recommended setting, but 'simple' is default for total
backwards compatibility.
(Will Johnson via hossman)

17. SOLR-248: Added CapitalizationFilterFactory that creates tokens with

normalized capitalization. This filter is useful for facet display,
but will not work with a prefix query. (ryan)
SOLR-468: Change to the semantics to keep the original token, not the
token in the Map. Also switched to use Lucene's new reusable token
capabilities. (gsingers)

18. SOLR-307: Added NGramFilterFactory and EdgeNGramFilterFactory.

(Thomas Peuss via Otis Gospodnetic)

19. SOLR-305: analysis.jsp can be given a fieldtype instead of a field

name. (hossman)

20. SOLR-102: Added RegexFragmenter, which splits text for highlighting

based on a given pattern. (klaas)

21. SOLR-258: Date Faceting added to SimpleFacets. Facet counts

computed for ranges of size facet.date.gap (a DateMath expression)
between facet.date.start and facet.date.end. (hossman)

22. SOLR-196: A PHP serialized "phps" response writer that returns a

serialized array that can be used with the PHP function unserialize,
and a PHP response writer "php" that may be used by eval.
(Nick Jenkin, Paul Borgermans, Pieter Berkel via yonik)

23. SOLR-308: A new UUIDField class which accepts UUID string values,
as well as the special value of "NEW" which triggers generation of
a new random UUID.
(Thomas Peuss via hossman)

24. SOLR-349: New FunctionQuery functions: sum, product, div, pow, log,
sqrt, abs, scale, map. Constants may now be used as a value source.
(yonik)

25. SOLR-359: Add field type className to Luke response, and enabled access
to the detailed field information from the solrj client API.
 (Grant Ingersoll via ehatcher)

26. SOLR-334: Pluggable query parsers. Allows specification of query

type and arguments as a prefix on a query string. (yonik)

27. SOLR-351: External Value Source. An external file may be used

to specify the values of a field, currently usable as
a ValueSource in a FunctionQuery. (yonik)

28. SOLR-395: Many new features for the spell checker implementation, including
an extended response mode with much richer output, multi-word spell checking,
and a bevy of new and renamed options (see the wiki).
(Mike Krimerman, Scott Taber via klaas).

29. SOLR-408: Added PingRequestHandler and deprecated

SolrCore.getPingQueryRequest().
Ping requests should be configured using standard RequestHandler syntax in
solrconfig.xml rather then using the <pingQuery></pingQuery> syntax.
(Karsten Sperling via ryan)

30. SOLR-281: Added a 'Search Component' interface and converted

StandardRequestHandler
and DisMaxRequestHandler to use this framework.
(Sharad Agarwal, Henri Biestro, yonik, ryan)

31. SOLR-176: Add detailed timing data to query response output. The SearchHandler
interface now returns how long each section takes. (klaas)

32. SOLR-414: Plugin initialization now supports SolrCore and ResourceLoader

"Aware"
plugins. Plugins that implement SolrCoreAware or ResourceLoaderAware are
informed about the SolrCore/ResourceLoader. (Henri Biestro, ryan)

33. SOLR-350: Support multiple SolrCores running in the same solr instance and
allows
runtime runtime management for any running SolrCore. If a solr.xml file exists
in solr.home, this file is used to instanciate multiple cores and enables
runtime
core manipulation. For more informaion see:
http://wiki.apache.org/solr/CoreAdmin
(Henri Biestro, ryan)

34. SOLR-447: Added an single request handler that will automatically register all
standard admin request handlers. This replaces the need to register (and
maintain)
the set of admin request handlers. Assuming solrconfig.xml includes:
<requestHandler name="/admin/"
class="org.apache.solr.handler.admin.AdminHandlers" />
This will register:
Luke/SystemInfo/PluginInfo/ThreadDump/PropertiesRequestHandler.
(ryan)

35. SOLR-142: Added RawResponseWriter and ShowFileRequestHandler. This returns

config
files directly. If AdminHandlers are configured, this will be added
automatically.
The jsp files /admin/get-file.jsp and /admin/raw-schema.jsp have been
deprecated.
The deprecated <admin><gettableFiles> will be automatically registered with
 a ShowFileRequestHandler instance for backwards compatibility. (ryan)

36. SOLR-446: TextResponseWriter can write SolrDocuments and SolrDocumentLists the

same way it writes Document and DocList. (yonik, ryan)

37. SOLR-418: Adding a query elevation component. This is an optional component to

elevate some documents to the top positions (or exclude them) for a given
query.
(ryan)

38. SOLR-478: Added ability to get back unique key information from the
LukeRequestHandler.
(gsingers)

39. SOLR-127: HTTP Caching awareness. Solr now recognizes HTTP Request
headers related to HTTP Caching (see RFC 2616 sec13) and will respond
with "304 Not Modified" when appropriate. New options have been added
to solrconfig.xml to influence this behavior.
(Thomas Peuss via hossman)

40. SOLR-303: Distributed Search over HTTP. Specification of shards

argument causes Solr to query those shards and merge the results
into a single response. Querying, field faceting (sorted only),
query faceting, highlighting, and debug information are supported
in distributed mode.
(Sharad Agarwal, Patrick O'Leary, Sabyasachi Dalal, Stu Hood,
Jayson Minard, Lars Kotthoff, ryan, yonik)

41. SOLR-356: Pluggable functions (value sources) that allow

registration of new functions via solrconfig.xml
(Doug Daniels via yonik)

42. SOLR-494: Added cool admin Ajaxed schema explorer.

(Greg Ludington via ehatcher)

43. SOLR-497: Added date faceting to the QueryResponse in SolrJ

and QueryResponseTest (Shalin Shekhar Mangar via gsingers)

44. SOLR-486: Binary response format, faster and smaller

than XML and JSON response formats (use wt=javabin).
BinaryResponseParser for utilizing the binary format via SolrJ
and is now the default.
(Noble Paul, yonik)

45. SOLR-521: StopFilterFactory support for "enablePositionIncrements"

(Walter Ferrara via hossman)

46. SOLR-557: Added SolrCore.getSearchComponents() to return an unmodifiable Map.

(gsingers)

47. SOLR-516: Added hl.maxAlternateFieldLength parameter, to set max length for

hl.alternateField
(Koji Sekiguchi via klaas)

48. SOLR-319: Changed SynonymFilterFactory to "tokenize" synonyms file.

To use a tokenizer, specify "tokenizerFactory" attribute in <filter>.
For example:
<tokenizer class="solr.CJKTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" expand="true"
 ignoreCase="true" tokenizerFactory="solr.CJKTokenizerFactory"/>
(koji)

49. SOLR-515: Added SimilarityFactory capability to schema.xml,

making config file parameters usable in the construction of
the global Lucene Similarity implementation.
(ehatcher)

50. SOLR-536: Add a DocumentObjectBinder to solrj that converts Objects to and

from SolrDocuments. (Noble Paul via ryan)

51. SOLR-595: Add support for Field level boosting in the MoreLikeThis Handler.
(Tom Morton, gsingers)

52. SOLR-572: Added SpellCheckComponent and org.apache.solr.spelling package to

support more spell
checking functionality. Also includes ability to add your own SolrSpellChecker
implementation that
plugs in. See http://wiki.apache.org/solr/SpellCheckComponent for more details
(Shalin Shekhar Mangar, Bojan Smid, gsingers)

53. SOLR-679: Added accessor methods to Lucene based spell checkers (gsingers)

54. SOLR-423: Added Request Handler close hook notification so that RequestHandlers
can be notified
when a core is closing. (gsingers, ryan)

55. SOLR-603: Added ability to partially optimize. (gsingers)

56. SOLR-483: Add byte/short sorting support (gsingers)

57. SOLR-14: Add preserveOriginal flag to WordDelimiterFilter

(Geoffrey Young, Trey Hyde, Ankur Madnani, yonik)

58. SOLR-502: Add search timeout support. (Sean Timm via yonik)

59. SOLR-605: Add the ability to register callbacks programatically (ryan, Noble
Paul)

60. SOLR-610: hl.maxAnalyzedChars can be -1 to highlight everything (Lars Kotthoff

via klaas)

61. SOLR-522: Make analysis.jsp show payloads. (Tricia Williams via yonik)

62. SOLR-611: Expose sort_values returned by QueryComponent in SolrJ's

QueryResponse
(Dan Rosher via shalin)

63. SOLR-256: Support exposing Solr statistics through JMX (Sharad Agrawal, shalin)

64. SOLR-666: Expose warmup time in statistics for SolrIndexSearcher and LRUCache
(shalin)

65. SOLR-663: Allow multiple files for stopwords, keepwords, protwords and synonyms
(Otis Gospodnetic, shalin)

66. SOLR-469: Added DataImportHandler as a contrib project which makes indexing

data from Databases,
XML files and HTTP data sources into Solr quick and easy. Includes API and
implementations for
supporting multiple data sources, processors and transformers for importing
data. Supports full
data imports as well as incremental (delta) indexing. See
http://wiki.apache.org/solr/DataImportHandler
for more details. (Noble Paul, shalin)

67. SOLR-622: SpellCheckComponent supports auto-loading indices on startup and

optionally, (re)builds
indices on newSearcher event, if configured in solrconfig.xml (shalin)

68. SOLR-554: Hierarchical JDK log level selector for SOLR Admin replaces
logging.jsp
(Sean Timm via shalin)

69. SOLR-506: Emitting HTTP Cache headers can be enabled or disabled through
configuration on a
per-handler basis (shalin)

70. SOLR-716: Added support for properties in configuration files. Properties can
be specified in
solr.xml and can be used in solrconfig.xml and schema.xml (Henri Biestro,
hossman, ryan, shalin)

71. SOLR-1129 : Support binding dynamic fields to beans in SolrJ (Avlesh Singh ,
noble)

72. SOLR-920 : Cache and reuse IndexSchema . A new attribute added in solr.xml
called 'shareSchema' (noble)

73. SOLR-700: DIH: Allow configurable locales through a locale attribute in

fields for NumberFormatTransformer. (Stefan Oestreicher, shalin)

Changes in runtime behavior

1. SOLR-559: use Lucene updateDocument, deleteDocuments methods. This
removes the maxBufferedDeletes parameter added by SOLR-310 as Lucene
now manages the deletes. This provides slightly better indexing
performance and makes overwrites atomic, eliminating the possibility of
a crash causing duplicates. (yonik)

2. SOLR-689 / SOLR-695: If you have used "MultiCore" functionality in an

unreleased
version of 1.3-dev, many classes and configs have been renamed for the official
1.3 release. Speciffically, solr.xml has replaced multicore.xml, and uses a
slightly
different syntax. The solrj classes: MultiCore{Request/Response/Params} have
been
renamed: CoreAdmin{Request/Response/Params} (hossman, ryan, Henri Biestro)

3. SOLR-647: reference count the SolrCore uses to prevent a premature

close while a core is still in use. (Henri Biestro, Noble Paul, yonik)

4. SOLR-737: SolrQueryParser now uses a ConstantScoreQuery for wildcard

queries that prevent an exception from being thrown when the number
of matching terms exceeds the BooleanQuery clause limit. (yonik)

Optimizations
1. SOLR-276: improve JSON writer speed. (yonik)
 2. SOLR-310: bound and reduce memory usage by providing <maxBufferedDeletes>
parameter,
which flushes deleted without forcing the user to use <commit/> for this
purpose.
(klaas)

3. SOLR-348: short-circuit faceting if less than mincount docs match. (yonik)

4. SOLR-354: Optimize removing all documents. Now when a delete by query

of *:* is issued, the current index is removed. (yonik)

5. SOLR-377: Speed up response writers. (yonik)

6. SOLR-342: Added support into the SolrIndexWriter for using several new
features of the new
LuceneIndexWriter, including: setRAMBufferSizeMB(), setMergePolicy(),
setMergeScheduler.
Also, added support to specify Lucene's autoCommit functionality (not to be
confused with Solr's
similarily named autoCommit functionality) via the <luceneAutoCommit> config.
item. See the test
and example solrconfig.xml <indexDefaults> section for usage. Performance
during indexing should
be significantly increased by moving up to 2.3 due to Lucene's new indexing
capabilities.
Furthermore, the setRAMBufferSizeMB makes it more logical to decide on tuning
factors related to
indexing. For best performance, leave the mergePolicy and mergeScheduler as
the defaults and set
ramBufferSizeMB instead of maxBufferedDocs. The best value for this depends on
the types of
documents in use. 32 should be a good starting point, but reports have shown
up to 48 MB provides
good results. Note, it is acceptable to set both ramBufferSizeMB and
maxBufferedDocs, and Lucene
will flush based on whichever limit is reached first. (gsingers)

7. SOLR-330: Converted TokenStreams to use Lucene's new char array based

capabilities. (gsingers)

8. SOLR-624: Only take snapshots if there are differences to the index (Richard
Trey Hyde via gsingers)

9. SOLR-587: Delete by Query performance greatly improved by using

new underlying Lucene IndexWriter implementation. (yonik)

10. SOLR-730: Use read-only IndexReaders that don't synchronize

isDeleted(). This will speed up function queries and *:* queries
as well as improve their scalability on multi-CPU systems.
(Mark Miller via yonik)

Bug Fixes
1. Make TextField respect sortMissingFirst and sortMissingLast fields.
(J.J. Larrea via yonik)

2. autoCommit/maxDocs was not working properly when large autoCommit/maxTime

was specified (klaas)

3. SOLR-283: autoCommit was not working after delete. (ryan)

 4. SOLR-286: ContentStreamBase was not using default encoding for getBytes()
(Toru Matsuzawa via ryan)

5. SOLR-292: Fix MoreLikeThis facet counting. (Pieter Berkel via ryan)

6. SOLR-297: Fix bug in RequiredSolrParams where requiring a field

specific param would fail if a general default value had been supplied.
(hossman)

7. SOLR-331: Fix WordDelimiterFilter handling of offsets for synonyms or

other injected tokens that can break highlighting. (yonik)

8. SOLR-282: Snapshooter does not work on Solaris and OS X since the cp command
there does not have the -l option. Also updated commit/optimize related
scripts to handle both old and new response format. (bill)

9. SOLR-294: Logging of elapsed time broken on Solaris because the date command
there does not support the %s output format. (bill)

10. SOLR-136: Snappuller - "date -d" and locales don't mix. (Jürgen Hermann via
bill)

11. SOLR-333: Changed distributiondump.jsp to use Solr HOME instead of CWD to set
path.

12. SOLR-393: Removed duplicate contentType from raw-schema.jsp. (bill)

13. SOLR-413: Requesting a large numbers of documents to be returned (limit)

can result in an out-of-memory exception, even for a small index. (yonik)

14. The CSV loader incorrectly threw an exception when given

header=true (the default). (ryan, yonik)

15. SOLR-449: the python and ruby response writers are now able to correctly
output NaN and Infinity in their respective languages. (klaas)

16. SOLR-42: HTMLStripReader tokenizers now preserve correct source

offsets for highlighting. (Grant Ingersoll via yonik)

17. SOLR-481: Handle UnknownHostException in _info.jsp (gsingers)

18. SOLR-324: Add proper support for Long and Doubles in sorting, etc. (gsingers)

19. SOLR-496: Cache-Control max-age changed to Long so Expires

calculation won't cause overflow. (Thomas Peuss via hossman)

20. SOLR-535: Fixed typo (Tokenzied -> Tokenized) in schema.jsp (Thomas Peuss via
billa)

21. SOLR-529: Better error messages from SolrQueryParser when field isn't
specified and there is no defaultSearchField in schema.xml
(Lars Kotthoff via hossman)

22. SOLR-530: Better error messages/warnings when parsing schema.xml:

field using bogus fieldtype and multiple copyFields to a non-multiValue
field. (Shalin Shekhar Mangar via hossman)

23. SOLR-528: Better error message when defaultSearchField is bogus or not

 indexed. (Lars Kotthoff via hossman)

24. SOLR-533: Fixed tests so they don't use hardcoded port numbers.
(hossman)

25. SOLR-400: SolrExceptionTest should now handle using OpenDNS as a DNS provider
(gsingers)

26. SOLR-541: Legacy XML update support (provided by SolrUpdateServlet

when no RequestHandler is mapped to "/update") now logs error correctly.
(hossman)

27. SOLR-267: Changed logging to report number of hits, and also provide a
mechanism to add log
messages to be output by the SolrCore via a NamedList toLog member variable.
(Will Johnson, yseeley, gsingers)

- SOLR-267: Removed adding values to the HTTP headers in SolrDispatchFilter

(gsingers)

28. SOLR-509: Moved firstSearcher event notification to the end of the SolrCore
constructor
(Koji Sekiguchi via gsingers)

29. SOLR-470, SOLR-552, SOLR-544, SOLR-701: Multiple fixes to DateField

regarding lenient parsing of optional milliseconds, and correct
formating using the canonical representation. LegacyDateField has
been added for people who have come to depend on the existing
broken behavior. (hossman, Stefan Oestreicher)

30. SOLR-539: Fix for non-atomic long counters and a cast fix to avoid divide
by zero. (Sean Timm via Otis Gospodnetic)

31. SOLR-514: Added explicit media-type with UTF* charset to *.xsl files that
don't already have one. (hossman)

32. SOLR-505: Give RequestHandlers the possiblity to suppress the generation

of HTTP caching headers. (Thomas Peuss via Otis Gospodnetic)

33. SOLR-553: Handle highlighting of phrase terms better when

hl.usePhraseHighligher=true URL param is used.
(Bojan Smid via Otis Gospodnetic)

34. SOLR-590: Limitation in pgrep on Linux platform breaks script-utils fixUser.

(Hannes Schmidt via billa)

35. SOLR-597: SolrServlet no longer "caches" SolrCore. This was causing

problems in Resin, and could potentially cause problems for customized
usages of SolrServlet.

36. SOLR-585: Now sets the QParser on the ResponseBuilder (gsingers)

37. SOLR-604: If the spellchecking path is relative, make it relative to the Solr
Data Directory.
(Shalin Shekhar Mangar via gsingers)

38. SOLR-584: Make stats.jsp and stats.xsl more robust.

(Yousef Ourabi and hossman)
39. SOLR-443: SolrJ: Declare UTF-8 charset on POSTed parameters
to avoid problems with servlet containers that default to latin-1
and allow switching of the exact POST mechanism for parameters
via useMultiPartPost in CommonsHttpSolrServer.
(Lars Kotthoff, Andrew Schurman, ryan, yonik)

40. SOLR-556: multi-valued fields always highlighted in disparate snippets

(Lars Kotthoff via klaas)

41. SOLR-501: Fix admin/analysis.jsp UTF-8 input for some other servlet
containers such as Tomcat. (Hiroaki Kawai, Lars Kotthoff via yonik)

42. SOLR-616: SpellChecker accuracy configuration is not applied for

FileBasedSpellChecker.
Apply it for FileBasedSpellChecker and IndexBasedSpellChecker both.
(shalin)

43. SOLR-648: SpellCheckComponent throws NullPointerException on using spellcheck.q

request
parameter after restarting Solr, if reload is called but build is not called.
(Jonathan Lee, shalin)

44. SOLR-598: DebugComponent now always occurs last in the SearchHandler list
unless the
components are explicitly declared. (gsingers)

45. SOLR-676: DataImportHandler should use UpdateRequestProcessor API instead of

directly
using UpdateHandler. (shalin)

46. SOLR-696: Fixed bug in NamedListCodec in regards to serializing Iterable

objects. (gsingers)

47. SOLR-669: snappuler fix for FreeBSD/Darwin (Richard "Trey" Hyde via Otis
Gospodnetic)

48. SOLR-606: Fixed spell check collation offset issue. (Stefan Oestreicher ,
Geoffrey Young, gsingers)

49. SOLR-589: Improved handling of badly formated query strings (Sean Timm via Otis
Gospodnetic)

50. SOLR-749: Allow QParser and ValueSourceParsers to be extended with same name
(hossman, gsingers)

51. SOLR-704: DIH NumberFormatTransformer can silently ignore part of the

string while parsing. Now it tries to use the complete string for parsing.
Failure to do so will result in an exception.
(Stefan Oestreicher via shalin)

52. SOLR-729: DIH Context.getDataSource(String) gives current entity's

DataSource instance regardless of argument. (Noble Paul, shalin)

53. SOLR-726: DIH: Jdbc Drivers and DataSources fail to load if placed in
multicore sharedLib or core's lib directory.
(Walter Ferrara, Noble Paul, shalin)

Other Changes
1. SOLR-135: Moved common classes to org.apache.solr.common and altered the
 build scripts to make two jars: apache-solr-1.3.jar and
apache-solr-1.3-common.jar. This common.jar can be used in client code;
It does not have lucene or junit dependencies. The original classes
have been replaced with a @Deprecated extended class and are scheduled
to be removed in a later release. While this change does not affect API
compatibility, it is recommended to update references to these
deprecated classes. (ryan)

2. SOLR-268: Tweaks to post.jar so it prints the error message from Solr.

(Brian Whitman via hossman)

3. Upgraded to Lucene 2.2.0; June 18, 2007.

4. SOLR-215: Static access to SolrCore.getSolrCore() and SolrConfig.config

have been deprecated in order to support multiple loaded cores.
(Henri Biestro via ryan)

5. SOLR-367: The create method in all TokenFilter and Tokenizer Factories

provided by Solr now declare their specific return types instead of just
using "TokenStream" (hossman)

6. SOLR-396: Hooks add to build system for automatic generation of (stub)

Tokenizer and TokenFilter Factories.
Also: new Factories for all Tokenizers and TokenFilters provided by the
lucene-analyzers-2.2.0.jar -- includes support for German, Chinese,
Russan, Dutch, Greek, Brazilian, Thai, and French. (hossman)

7. Upgraded to commons-CSV r609327, which fixes escaping bugs and

introduces new escaping and whitespace handling options to
increase compatibility with different formats. (yonik)

8. Upgraded to Lucene 2.3.0; Jan 23, 2008.

9. SOLR-451: Changed analysis.jsp to use POST instead of GET, also made the input
area a
bit bigger (gsingers)

10. Upgrade to Lucene 2.3.1

11. SOLR-531: Different exit code for rsyncd-start and snappuller if disabled
(Thomas Peuss via billa)

12. SOLR-550: Clarified DocumentBuilder addField javadocs (gsingers)

13. Upgrade to Lucene 2.3.2

14. SOLR-518: Changed luke.xsl to use divs w/css for generating histograms
instead of SVG (Thomas Peuss via hossman)

15. SOLR-592: Added ShardParams interface and changed several string literals
to references to constants in CommonParams.
(Lars Kotthoff via Otis Gospodnetic)

16. SOLR-520: Deprecated unused LengthFilter since already core in

Lucene-Java (hossman)

17. SOLR-645: Refactored SimpleFacetsTest (Lars Kotthoff via hossman)

18. SOLR-591: Changed Solrj default value for facet.sort to true (Lars Kotthoff via
Shalin)

19. Upgraded to Lucene 2.4-dev (r669476) to support SOLR-572 (gsingers)

20. SOLR-636: Improve/simplify example configs; and make index.jsp

links more resilient to configs loaded via an InputStream
(Lars Kotthoff, hossman)

21. SOLR-682: Scripts now support FreeBSD (Richard Trey Hyde via gsingers)

22. SOLR-489: Added in deprecation comments. (Sean Timm, Lars Kothoff via
gsingers)

23. SOLR-692: Migrated to stable released builds of StAX API 1.0.1 and StAX 1.2.0
(shalin)
24. Upgraded to Lucene 2.4-dev (r686801) (yonik)
25. Upgraded to Lucene 2.4-dev (r688745) 27-Aug-2008 (yonik)
26. Upgraded to Lucene 2.4-dev (r691741) 03-Sep-2008 (yonik)
27. Replaced the StAX reference implementation with the geronimo
StAX API jar, and the Woodstox StAX implementation. (yonik)

Build
1. SOLR-411. Changed the names of the Solr JARs to use the defacto standard JAR
names based on
project-name-version.jar. This yields, for example:
apache-solr-common-1.3-dev.jar
apache-solr-solrj-1.3-dev.jar
apache-solr-1.3-dev.jar

2. SOLR-479: Added clover code coverage targets for committers and the nightly
build. Requires
the Clover library, as licensed to Apache and only available privately. To
run:
ant -Drun.clover=true clean clover test generate-clover-reports

3. SOLR-510: Nightly release includes client sources. (koji)

4. SOLR-563: Modified the build process to build contrib projects

(Shalin Shekhar Mangar via Otis Gospodnetic)

5. SOLR-673: Modify build file to create javadocs for core, solrj, contrib and
"all inclusive" (shalin)

6. SOLR-672: Nightly release includes contrib sources. (Jeremy Hinegardner,

shalin)

7. SOLR-586: Added ant target and POM files for building maven artifacts of the
Solr core, common,
client and contrib. The target can publish artifacts with source and
javadocs.
(Spencer Crissman, Craig McClanahan, shalin)

================== Release 1.2 ==================

Upgrading from Solr 1.1

-------------------------------------
IMPORTANT UPGRADE NOTE: In a master/slave configuration, all searchers/slaves
should be upgraded before the master! If the master were to be updated
first, the older searchers would not be able to read the new index format.
Older Apache Solr installations can be upgraded by replacing
the relevant war file with the new version. No changes to configuration
files should be needed.

This version of Solr contains a new version of Lucene implementing

an updated index format. This version of Solr/Lucene can still read
and update indexes in the older formats, and will convert them to the new
format on the first index change. One change in the new index format
is that all "norms" are kept in a single file, greatly reducing the number
of files per segment. Users of compound file indexes will want to consider
converting to the non-compound format for faster indexing and slightly better
search concurrency.

The JSON response format for facets has changed to make it easier for
clients to retain sorted order. Use json.nl=map explicitly in clients
to get the old behavior, or add it as a default to the request handler
in solrconfig.xml

The Lucene based Solr query syntax is slightly more strict.

A ':' in a field value must be escaped or the whole value must be quoted.

The Solr "Request Handler" framework has been updated in two key ways:
First, if a Request Handler is registered in solrconfig.xml with a name
starting with "/" then it can be accessed using path-based URL, instead of
using the legacy "/select?qt=name" URL structure. Second, the Request
Handler framework has been extended making it possible to write Request
Handlers that process streams of data for doing updates, and there is a
new-style Request Handler for XML updates given the name of "/update" in
the example solrconfig.xml. Existing installations without this "/update"
handler will continue to use the old update servlet and should see no
changes in behavior. For new-style update handlers, errors are now
reflected in the HTTP status code, Content-type checking is more strict,
and the response format has changed and is controllable via the wt
parameter.

Detailed Change List

--------------------

New Features
1. SOLR-82: Default field values can be specified in the schema.xml.
(Ryan McKinley via hossman)

2. SOLR-89: Two new TokenFilters with corresponding Factories...

* TrimFilter - Trims leading and trailing whitespace from Tokens
* PatternReplaceFilter - applies a Pattern to each token in the
stream, replacing match occurances with a specified replacement.
(hossman)

3. SOLR-91: allow configuration of a limit of the number of searchers

that can be warming in the background. This can be used to avoid
out-of-memory errors, or contention caused by more and more searchers
warming in the background. An error is thrown if the limit specified
by maxWarmingSearchers in solrconfig.xml is exceeded. (yonik)

4. SOLR-106: New faceting parameters that allow specification of a

minimum count for returned facets (facet.mincount), paging through facets
 (facet.offset, facet.limit), and explicit sorting (facet.sort).
facet.zeros is now deprecated. (yonik)

5. SOLR-80: Negative queries are now allowed everywhere. Negative queries

are generated and cached as their positive counterpart, speeding
generation and generally resulting in smaller sets to cache.
Set intersections in SolrIndexSearcher are more efficient,
starting with the smallest positive set, subtracting all negative
sets, then intersecting with all other positive sets. (yonik)

6. SOLR-117: Limit a field faceting to constraints with a prefix specified

by facet.prefix or f.<field>.facet.prefix. (yonik)

7. SOLR-107: JAVA API: Change NamedList to use Java5 generics

and implement Iterable<Map.Entry> (Ryan McKinley via yonik)

8. SOLR-104: Support for "Update Plugins" -- RequestHandlers that want

access to streams of data for doing updates. ContentStreams can come
from the raw POST body, multi-part form data, or remote URLs.
Included in this change is a new SolrDispatchFilter that allows
RequestHandlers registered with names that begin with a "/" to be
accessed using a URL structure based on that name.
(Ryan McKinley via hossman)

9. SOLR-126: DirectUpdateHandler2 supports autocommitting after a specified time

(in ms), using <autoCommit><maxTime>10000</maxTime></autoCommit>.
(Ryan McKinley via klaas).

10. SOLR-116: IndexInfoRequestHandler added. (Erik Hatcher)

11. SOLR-79: Add system property ${<sys.prop>[:<default>]} substitution for

configuration files loaded, including schema.xml and solrconfig.xml.
(Erik Hatcher with inspiration from Andrew Saar)

12. SOLR-149: Changes to make Solr more easily embeddable, in addition

to logging which request handler handled each request.
(Ryan McKinley via yonik)

13. SOLR-86: Added standalone Java-based command-line updater.

(Erik Hatcher via Bertrand Delecretaz)

14. SOLR-152: DisMaxRequestHandler now supports configurable alternate

behavior when q is not specified. A "q.alt" param can be specified
using SolrQueryParser syntax as a mechanism for specifying what query
the dismax handler should execute if the main user query (q) is blank.
(Ryan McKinley via hossman)

15. SOLR-158: new "qs" (Query Slop) param for DisMaxRequestHandler

allows for specifying the amount of default slop to use when parsing
explicit phrase queries from the user.
(Adam Hiatt via hossman)

16. SOLR-81: SpellCheckerRequestHandler that uses the SpellChecker from

the Lucene contrib.
(Otis Gospodnetic and Adam Hiatt)

17. SOLR-182: allow lazy loading of request handlers on first request.

(Ryan McKinley via yonik)
18. SOLR-81: More SpellCheckerRequestHandler enhancements, inlcluding
support for relative or absolute directory path configurations, as
well as RAM based directory. (hossman)

19. SOLR-197: New parameters for input: stream.contentType for specifying

or overriding the content type of input, and stream.file for reading
local files. (Ryan McKinley via yonik)

20. SOLR-66: CSV data format for document additions and updates. (yonik)

21. SOLR-184: add echoHandler=true to responseHeader, support echoParams=all

(Ryan McKinley via ehatcher)

22. SOLR-211: Added a regex PatternTokenizerFactory. This extracts tokens

from the input string using a regex Pattern. (Ryan McKinley)

23. SOLR-162: Added a "Luke" request handler and other admin helpers.
This exposes the system status through the standard requestHandler
framework. (ryan)

24. SOLR-212: Added a DirectSolrConnection class. This lets you access

solr using the standard request/response formats, but does not require
an HTTP connection. It is designed for embedded applications. (ryan)

25. SOLR-204: The request dispatcher (added in SOLR-104) can handle

calls to /select. This offers uniform error handling for /update and
/select. To enable this behavior, you must add:
<requestDispatcher handleSelect="true" > to your solrconfig.xml
See the example solrconfig.xml for details. (ryan)

26. SOLR-170: StandardRequestHandler now supports a "sort" parameter.

Using the ';' syntax is still supported, but it is recommended to
transition to the new syntax. (ryan)

27. SOLR-181: The index schema now supports "required" fields. Attempts
to add a document without a required field will fail, returning a
descriptive error message. By default, the uniqueKey field is
a required field. This can be disabled by setting required=false
in schema.xml. (Greg Ludington via ryan)

28. SOLR-217: Fields configured in the schema to be neither indexed or

stored will now be quietly ignored by Solr when Documents are added.
The example schema has a comment explaining how this can be used to
ignore any "unknown" fields.
(Will Johnson via hossman)

29. SOLR-227: If schema.xml defines multiple fieldTypes, fields, or

dynamicFields with the same name, a severe error will be logged rather
then quietly continuing. Depending on the <abortOnConfigurationError>
settings, this may halt the server. Likewise, if solrconfig.xml
defines multiple RequestHandlers with the same name it will also add
an error. (ryan)

30. SOLR-226: Added support for dynamic field as the destination of a

copyField using glob (*) replacement. (ryan)

31. SOLR-224: Adding a PhoneticFilterFactory that uses apache commons codec

language encoders to build phonetically similar tokens. This currently
supports: DoubleMetaphone, Metaphone, Soundex, and RefinedSoundex (ryan)
32. SOLR-199: new n-gram tokenizers available via NGramTokenizerFactory
and EdgeNGramTokenizerFactory. (Adam Hiatt via yonik)

33. SOLR-234: TrimFilter can update the Token's startOffset and endOffset
if updateOffsets="true". By default the Token offsets are unchanged.
(ryan)

34. SOLR-208: new example_rss.xsl and example_atom.xsl to provide more

examples for people about the Solr XML response format and how they
can transform it to suit different needs.
(Brian Whitman via hossman)

35. SOLR-249: Deprecated SolrException( int, ... ) constructors in favor

of constructors that takes an ErrorCode enum. This will ensure that
all SolrExceptions use a valid HTTP status code. (ryan)

36. SOLR-386: Abstracted SolrHighlighter and moved existing implementation

to DefaultSolrHighlighter. Adjusted SolrCore and solrconfig.xml so
that highlighter is configurable via a class attribute. Allows users
to use their own highlighter implementation. (Tricia Williams via klaas)

Changes in runtime behavior

1. Highlighting using DisMax will only pick up terms from the main
user query, not boost or filter queries (klaas).

2. SOLR-125: Change default of json.nl to flat, change so that

json.nl only affects items where order matters (facet constraint
listings). Fix JSON output bug for null values. Internal JAVA API:
change most uses of NamedList to SimpleOrderedMap. (yonik)

3. A new method "getSolrQueryParser" has been added to the IndexSchema

class for retrieving a new SolrQueryParser instance with all options
specified in the schema.xml's <solrQueryParser> block set. The
documentation for the SolrQueryParser constructor and its use of
IndexSchema have also been clarified.
(Erik Hatcher and hossman)

4. DisMaxRequestHandler's bq, bf, qf, and pf parameters can now accept

multiple values (klaas).

5. Query are re-written before highlighting is performed. This enables

proper highlighting of prefix and wildcard queries (klaas).

6. A meaningful exception is raised when attempting to add a doc missing

a unique id if it is declared in the schema and allowDups=false.
(ryan via klaas)

7. SOLR-183: Exceptions with error code 400 are raised when

numeric argument parsing fails. RequiredSolrParams class added
to facilitate checking for parameters that must be present.
(Ryan McKinley, J.J. Larrea via yonik)

8. SOLR-179: By default, solr will abort after any severe initialization

errors. This behavior can be disabled by setting:
<abortOnConfigurationError>false</abortOnConfigurationError>
in solrconfig.xml (ryan)

9. The example solrconfig.xml maps /update to XmlUpdateRequestHandler using

 the new request dispatcher (SOLR-104). This requires posted content to
have a valid contentType: curl -H 'Content-type:text/xml; charset=utf-8'
The response format matches that of /select and returns standard error
codes. To enable solr1.1 style /update, do not map "/update" to any
handler in solrconfig.xml (ryan)

10. SOLR-231: If a charset is not specified in the contentType,

ContentStream.getReader() will use UTF-8 encoding. (ryan)

11. SOLR-230: More options for post.jar to support stdin, xml on the
commandline, and defering commits. Tutorial modified to take
advantage of these options so there is no need for curl.
(hossman)

12. SOLR-128: Upgraded Jetty to the latest stable release 6.1.3 (ryan)

Optimizations
1. SOLR-114: HashDocSet specific implementations of union() and andNot()
for a 20x performance improvement for those set operations, and a new
hash algorithm speeds up exists() by 10% and intersectionSize() by 8%.
(yonik)

2. SOLR-115: Solr now uses BooleanQuery.clauses() instead of

BooleanQuery.getClauses() in any situation where there is no risk of
modifying the original query.
(hossman)

3. SOLR-221: Speed up sorted faceting on multivalued fields by ~60%

when the base set consists of a relatively large portion of the
index. (yonik)

4. SOLR-221: Added a facet.enum.cache.minDf parameter which avoids

using the filterCache for terms that match few documents, trading
decreased memory usage for increased query time. (yonik)

Bug Fixes
1. SOLR-87: Parsing of synonym files did not correctly handle escaped
whitespace such as \r\n\t\b\f. (yonik)

2. SOLR-92: DOMUtils.getText (used when parsing config files) did not

work properly with many DOM implementations when dealing with
"Attributes". (Ryan McKinley via hossman)

3. SOLR-9,SOLR-99: Tighten up sort specification error checking, throw

exceptions for missing sort specifications or a sort on a non-indexed
field. (Ryan McKinley via yonik)

4. SOLR-145: Fix for bug introduced in SOLR-104 where some Exceptions

were being ignored by all "out of the box" RequestHandlers. (hossman)

5. SOLR-166: JNDI solr.home code refactoring. SOLR-104 moved

some JNDI related code to the init method of a Servlet Filter -
according to the Servlet Spec, all Filter's should be initialized
prior to initializing any Servlets, but this is not the case in at
least one Servlet Container (Resin). This "bug fix" refactors
this JNDI code so that it should be executed the first time any
attempt is made to use the solr.home dir.
(Ryan McKinley via hossman)
 6. SOLR-173: Bug fix to SolrDispatchFilter to reduce "too many open
files" problem was that SolrDispatchFilter was not closing requests
when finished. Also modified ResponseWriters to only fetch a Searcher
reference if necessary for writing out DocLists.
(Ryan McKinley via hossman)

7. SOLR-168: Fix display positioning of multiple tokens at the same

position in analysis.jsp (yonik)

8. SOLR-167: The SynonymFilter sometimes generated incorrect offsets when

multi token synonyms were mached in the source text. (yonik)

9. SOLR-188: bin scripts do not support non-default webapp names. Added "-U"
option to specify a full path to the update url, overriding the
"-h" (hostname), "-p" (port) and "-w" (webapp name) parameters.
(Jeff Rodenburg via billa)

10. SOLR-198: RunExecutableListener always waited for the process to

finish, even when wait="false" was set. (Koji Sekiguchi via yonik)

11. SOLR-207: Changed distribution scripts to remove recursive find

and avoid use of "find -maxdepth" on platforms where it is not
supported. (yonik)

12. SOLR-222: Changing writeLockTimeout in solrconfig.xml did not

change the effective timeout. (Koji Sekiguchi via yonik)

13. Changed the SOLR-104 RequestDispatcher so that /select?qt=xxx can not

access handlers that start with "/". This makes path based authentication
possible for path based request handlers. (ryan)

14. SOLR-214: Some servlet containers (including Tomcat and Resin) do not
obey the specified charset. Rather then letting the the container handle
it solr now uses the charset from the header contentType to decode posted
content. Using the contentType: "text/xml; charset=utf-8" will force
utf-8 encoding. If you do not specify a contentType, it will use the
platform default. (Koji Sekiguchi via ryan)

15. SOLR-241: Undefined system properties used in configuration files now

cause a clear message to be logged rather than an obscure exception thrown.
(Koji Sekiguchi via ehatcher)

Other Changes
1. Updated to Lucene 2.1

2. Updated to Lucene 2007-05-20_00-04-53

================== Release 1.1.0 ==================

Status
------
This is the first release since Solr joined the Incubator, and brings many
new features and performance optimizations including highlighting,
faceted browsing, and JSON/Python/Ruby response formats.

Upgrading from previous Solr versions

-------------------------------------
Older Apache Solr installations can be upgraded by replacing
the relevant war file with the new version. No changes to configuration
files are needed and the index format has not changed.

The default version of the Solr XML response syntax has been changed to 2.2.
Behavior can be preserved for those clients not explicitly specifying a
version by adding a default to the request handler in solrconfig.xml

By default, Solr will no longer use a searcher that has not fully warmed,
and requests will block in the meantime. To change back to the previous
behavior of using a cold searcher in the event there is no other
warm searcher, see the useColdSearcher config item in solrconfig.xml

The XML response format when adding multiple documents to the collection
in a single <add> command has changed to return a single <result>.

Detailed Change List

--------------------

New Features
1. added support for setting Lucene's positionIncrementGap
2. Admin: new statistics for SolrIndexSearcher
3. Admin: caches now show config params on stats page
3. max() function added to FunctionQuery suite
4. postOptimize hook, mirroring the functionallity of the postCommit hook,
but only called on an index optimize.
5. Ability to HTTP POST query requests to /select in addition to HTTP-GET
6. The default search field may now be overridden by requests to the
standard request handler using the df query parameter. (Erik Hatcher)
7. Added DisMaxRequestHandler and SolrPluginUtils. (Chris Hostetter)
8. Support for customizing the QueryResponseWriter per request
(Mike Baranczak / SOLR-16 / hossman)
9. Added KeywordTokenizerFactory (hossman)
10. copyField accepts dynamicfield-like names as the source.
(Darren Erik Vengroff via yonik, SOLR-21)
11. new DocSet.andNot(), DocSet.andNotSize() (yonik)
12. Ability to store term vectors for fields. (Mike Klaas via yonik, SOLR-23)
13. New abstract BufferedTokenStream for people who want to write
Tokenizers or TokenFilters that require arbitrary buffering of the
stream. (SOLR-11 / yonik, hossman)
14. New RemoveDuplicatesToken - useful in situations where
synonyms, stemming, or word-deliminater-ing produce identical tokens at
the same position. (SOLR-11 / yonik, hossman)
15. Added highlighting to SolrPluginUtils and implemented in StandardRequestHandler
and DisMaxRequestHandler (SOLR-24 / Mike Klaas via hossman,yonik)
16. SnowballPorterFilterFactory language is configurable via the "language"
attribute, with the default being "English". (Bertrand Delacretaz via yonik,
SOLR-27)
17. ISOLatin1AccentFilterFactory, instantiates ISOLatin1AccentFilter to remove
accents.
(Bertrand Delacretaz via yonik, SOLR-28)
18. JSON, Python, Ruby QueryResponseWriters: use wt="json", "python" or "ruby"
(yonik, SOLR-31)
19. Make web admin pages return UTF-8, change Content-type declaration to include a
space between the mime-type and charset (Philip Jacob, SOLR-35)
20. Made query parser default operator configurable via schema.xml:
<solrQueryParser defaultOperator="AND|OR"/>
The default operator remains "OR".
21. JAVA API: new version of SolrIndexSearcher.getDocListAndSet() which takes
 flags (Greg Ludington via yonik, SOLR-39)
22. A HyphenatedWordsFilter, a text analysis filter used during indexing to rejoin
words that were hyphenated and split by a newline. (Boris Vitez via yonik,
SOLR-41)
23. Added a CompressableField base class which allows fields of derived types to
be compressed using the compress=true setting. The field type also gains the
ability to specify a size threshold at which field data is compressed.
(klaas, SOLR-45)
24. Simple faceted search support for fields (enumerating terms)
and arbitrary queries added to both StandardRequestHandler and
DisMaxRequestHandler. (hossman, SOLR-44)
25. In addition to specifying default RequestHandler params in the
solrconfig.xml, support has been added for configuring values to be
appended to the multi-val request params, as well as for configuring
invariant params that can not overridden in the query. (hossman, SOLR-46)
26. Default operator for query parsing can now be specified with q.op=AND|OR
from the client request, overriding the schema value. (ehatcher)
27. New XSLTResponseWriter does server side XSLT processing of XML Response.
In the process, an init(NamedList) method was added to QueryResponseWriter
which works the same way as SolrRequestHandler.
(Bertrand Delacretaz / SOLR-49 / hossman)
28. json.wrf parameter adds a wrapper-function around the JSON response,
useful in AJAX with dynamic script tags for specifying a JavaScript
callback function. (Bertrand Delacretaz via yonik, SOLR-56)
29. autoCommit can be specified every so many documents added (klaas, SOLR-65)
30. ${solr.home}/lib directory can now be used for specifying "plugin" jars
(hossman, SOLR-68)
31. Support for "Date Math" relative "NOW" when specifying values of a
DateField in a query -- or when adding a document.
(hossman, SOLR-71)
32. useColdSearcher control in solrconfig.xml prevents the first searcher
from being used before it's done warming. This can help prevent
thrashing on startup when multiple requests hit a cold searcher.
The default is "false", preventing use before warm. (yonik, SOLR-77)

Changes in runtime behavior

1. classes reorganized into different packages, package names changed to Apache
2. force read of document stored fields in QuerySenderListener
3. Solr now looks in ./solr/conf for config, ./solr/data for data
configurable via solr.solr.home system property
4. Highlighter params changed to be prefixed with "hl."; allow fragmentsize
customization and per-field overrides on many options
(Andrew May via klaas, SOLR-37)
5. Default param values for DisMaxRequestHandler should now be specified
using a '<lst name="defaults">...</lst>' init param, for backwards
compatability all init prams will be used as defaults if an init param
with that name does not exist. (hossman, SOLR-43)
6. The DisMaxRequestHandler now supports multiple occurances of the "fq"
param. (hossman, SOLR-44)
7. FunctionQuery.explain now uses ComplexExplanation to provide more
accurate score explanations when composed in a BooleanQuery.
(hossman, SOLR-25)
8. Document update handling locking is much sparser, allowing performance gains
through multiple threads. Large commits also might be faster (klaas, SOLR-65)
9. Lazy field loading can be enabled via a solrconfig directive. This will be
faster when
not all stored fields are needed from a document (klaas, SOLR-52)
10. Made admin JSPs return XML and transform them with new XSL stylesheets
(Otis Gospodnetic, SOLR-58)
11. If the "echoParams=explicit" request parameter is set, request parameters are
copied
to the output. In an XML output, they appear in new <lst name="params"> list
inside
the new <lst name="responseHeader"> element, which replaces the old
<responseHeader>.
Adding a version=2.1 parameter to the request produces the old format, for
backwards
compatibility (bdelacretaz and yonik, SOLR-59).

Optimizations
1. getDocListAndSet can now generate both a DocList and a DocSet from a
single lucene query.
2. BitDocSet.intersectionSize(HashDocSet) no longer generates an intermediate
set
3. OpenBitSet completed, replaces BitSet as the implementation for BitDocSet.
Iteration is faster, and BitDocSet.intersectionSize(BitDocSet) and unionSize
is between 3 and 4 times faster. (yonik, SOLR-15)
4. much faster unionSize when one of the sets is a HashDocSet: O(smaller_set_size)
5. Optimized getDocSet() for term queries resulting in a 36% speedup of
facet.field
queries where DocSets aren't cached (for example, if the number of terms in the
field
is larger than the filter cache.) (yonik)
6. Optimized facet.field faceting by as much as 500 times when the field has
a single token per document (not multiValued & not tokenized) by using the
Lucene FieldCache entry for that field to tally term counts. The first request
utilizing the FieldCache will take longer than subsequent ones.

Bug Fixes
1. Fixed delete-by-id for field types who's indexed form is different
from the printable form (mainly sortable numeric types).
2. Added escaping of attribute values in the XML response (Erik Hatcher)
3. Added empty extractTerms() to FunctionQuery to enable use in
a MultiSearcher (Yonik)
4. WordDelimiterFilter sometimes lost token positionIncrement information
5. Fix reverse sorting for fields were sortMissingFirst=true
(Rob Staveley, yonik)
6. Worked around a Jetty bug that caused invalid XML responses for fields
containing non ASCII chars. (Bertrand Delacretaz via yonik, SOLR-32)
7. WordDelimiterFilter can throw exceptions if configured with both
generate and catenate off. (Mike Klaas via yonik, SOLR-34)
8. Escape '>' in XML output (because ]]> is illegal in CharData)
9. field boosts weren't being applied and doc boosts were being applied to fields
(klaas)
10. Multiple-doc update generates well-formed xml (klaas, SOLR-65)
11. Better parsing of pingQuery from solrconfig.xml (hossman, SOLR-70)
12. Fixed bug with "Distribution" page introduced when Versions were
added to "Info" page (hossman)
13. Fixed HTML escaping issues with user input to analysis.jsp and action.jsp
(hossman, SOLR-74)

Other Changes
1. Upgrade to Lucene 2.0 nightly build 2006-06-22, lucene SVN revision 416224,
http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?
view=markup&pathrev=416224
2. Modified admin styles to improve display in Internet Explorer (Greg Ludington
via billa, SOLR-6)
3. Upgrade to Lucene 2.0 nightly build 2006-07-15, lucene SVN revision 422302,
 4. Included unique key field name/value (if available) in log message of add
(billa, SOLR-18)
5. Updated to Lucene 2.0 nightly build 2006-09-07, SVN revision 462111
6. Added javascript to catch empty query in admin query forms (Tomislav Nakic-
Alfirevic via billa, SOLR-48
7. blackslash escape * in ssh command used in snappuller for zsh compatibility,
SOLR-63
8. check solr return code in admin scripts, SOLR-62
9. Updated to Lucene 2.0 nightly build 2006-11-15, SVN revision 475069
10. Removed src/apps containing the legacy "SolrTest" app (hossman, SOLR-3)
11. Simplified index.jsp and form.jsp, primarily by removing/hiding XML
specific params, and adding an option to pick the output type. (hossman)
12. Added new numeric build property "specversion" to allow clean
MANIFEST.MF files (hossman)
13. Added Solr/Lucene versions to "Info" page (hossman)
14. Explicitly set mime-type of .xsl files in web.xml to
application/xslt+xml (hossman)
15. Config parsing should now work useing DOM Level 2 parsers -- Solr
previously relied on getTextContent which is a DOM Level 3 addition
(Alexander Saar via hossman, SOLR-78)

2006/01/17 Solr open sourced, moves to Apache Incubator