Weaviate v0.22.14 Release NotesRelease Date: 2020-07-14 // almost 2 years ago
💥 Breaking Changes
🆕 New Features
👌 Improved noise filtering in
Prior to this release the features relying on nearest neighbors (namely
_semanticPath) might have contained "noise words", i.e. concatenated words of random words with no easily recognizable meaning. These words are present in the Contextionary training space, but are extremely rare and therefore distributed seemingly randomly. As a result an "ordinary" result might have such a noise word as it's immediate neighbor.
To combat this noise, a neighbor filtering feature was introduced in the contextionary, which ignores words of the configured bottom percentile - ranked by occurrence in the respective training set. By default this value is set to the bottom 5th percentile. This setting can be overridden. To set another value, e.g. to ignore the bottom 10th percentile, provide the environment variable
contextionarycontainer. An example of this variable set correctly can be found here (english version) for example.
Note that this feature requires at least contextionary version
🔧 Check if required contextionary container is running and configured at startup (gh-1190)
🔧 Prior to this version, it was possible to run a new version of weaviate together with an outdated and incompatible contextionary version. This (invalid) combination would start up fine, but eventually lead to runtime errors when a contextionary method was called that was incompatible with the configured contextionary container.
🔧 This fix makes this impossible. If a contextionary version is configured that does not meet the required conditions, Weaviate will no longer start up. The logs will show one of the following options:
Option 1: Wrong version
FATA insufficient contextionary version, cannot start up action=startup_check_contextionary contextionaryVersion=en0.16.0-v0.4.14 requiredMinimumContextionaryVersion=0.4.15 exit status 1
Option 2: Happy path
INFO found a valid contextionary version action=startup_check_contextionary contextionaryVersion=en0.16.0-v0.4.15 requiredMinimumContextionaryVersion=0.4.15