All Versions
39
Latest Version
Avg Release Cycle
14 days
Latest Release
669 days ago

Changelog History
Page 1

  • v0.22.20 Changes

    November 27, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.20
    ๐Ÿ”– See also: example docker-compose files in English, Dutch, German, Czech, Italian. If you need to configure additional settings, you can also generate a custom docker-compose.yml file using the documentation.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    ๐Ÿ†• New classification (knn-only) ref meta fields added (#1244)
    ๐Ÿ‘€ See #1244 for details. When using the _classification underscore prop, the ref meta, i.e. the fields as part of each reference object contain new distances now. Note: Those distances will only be set if the object was affected by a kNN-classification. The new fields are overallCount, winningCount, losingCount, meanWinningDistance, meanLosingDistance, closestOverallDistance, closestWinningDistance and closestLosingDistance

    Standalone mode reaches feature parity with ES-based mode
    ๐Ÿš€ It is still possible to switch between modes. Standalone is not considered production ready yet, as some minor issues (see Milestone "Standalone") are not yet complete. ETA for a production-ready release - which removes all ES-features entirely - is v0.23.0 with ETA late December 2020.

    ๐Ÿ‘€ For details see #1265, #1249, #1271, #1272, #1281, #1250, #1291, #1278, #1286, #1308

    ๐Ÿ†• New Deprecations

    • winningDistance and losingDistance in _classification ref meta deprecated
      ๐Ÿ‘€ They have been replaced with meanLosingDistance and meanWinningDistance, see this deprecation notice for details.

    ๐Ÿ›  Fixes

    none

  • v0.22.19 Changes

    October 19, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.19
    ๐Ÿ”– See also: example docker-compose files in English, Dutch, German, Czech, Italian. If you need to configure additional settings, you can also generate a custom docker-compose.yml file using the documentation.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    • Introduce certainty in Get {} with explore set (#1258)
      ๐Ÿš€ Prior to this release the certainty (i.e. how close is a result to the search query) could only be set on Explore {}, but not on Get {} with an additional explore parameter set. This release introduces the _certainty underscore prop which introduces this ability

    ๐Ÿ›  Fixes

    • ๐Ÿ‘Œ Improvements in moveAwayFrom (#1267)
      ๐Ÿš€ This releases fixes issues where setting the moveAwayFrom parameter in Explore and Get{} with explore would lead to bad results that were in some cases not related to the original search term anymore. This fix uses an improved formula (see #1267 for details) to calculate the new position after moving.
  • v0.22.18 Changes

    September 28, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.18
    ๐Ÿ”– See also: example docker-compose files in English, Dutch, German, Czech, Italian. If you need to configure additional settings, you can also generate a custom docker-compose.yml file using the documentation.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    none

    ๐Ÿ›  Fixes

    • ๐Ÿ›  Fix error message on GraphQL {Explore} (gh-1256)
      accidentally introduced in 0.22.17
  • v0.22.17 Changes

    September 16, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.17
    ๐Ÿ”– See also: example docker-compose files in English, Dutch, German, Czech, Italian. If you need to configure additional settings, you can also generate a custom docker-compose.yml file using the documentation.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    none

    ๐Ÿ—„ Deprecations

    ๐Ÿ—„ Deprecate cardinality in schema property (#1240, #1142)
    ๐Ÿ—„ The cardinality field has become deprecated. There are no more restrictions. Regardless of the content of this field, reference properties can have 0..n references, primitive properties are always just a single property (i.e. they are either set or not set).

    ๐Ÿšš This change is non breaking as the field is not removed, but only ignored. If you explicitly still set a value for cardinality, weaviate will log a deprecation message to stdout.

    ๐Ÿ›  Fixes

    • โœ‚ Remove obsolete and unused telemetry logging (#1232)
    • ๐Ÿ›  Fix issue where periodic "telemetry logging" error message could occur
      ๐Ÿ”ง Related to the removal of this unused/obsolete feature there was a chance that such log messages would be printed depending on the configuration. In other words, it was possible to accidentally turn this feature on even though it shouldn't have been. This commit now removes the feature entirely, so it can't be accidentally turned on anymore.
  • v0.22.16 Changes

    September 15, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.16
    ๐Ÿณ See also: example docker-compose files in English, Dutch, German, Czech, Italian.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    ๐Ÿ”ง Specify configuration through environment variables (#1230)
    ๐Ÿš€ Prior to this release, Weaviate required a config file. That was not the cloud-native way to handle config. Furthermore, users who wanted to try out Weaviate using the docker-compose setup, had to download two files. Now all config can be managed through the docker-compose file itself.

    The following environment variables can be set:

    Variable Description Type Example Value
    ๐Ÿš€ ORIGIN Set the http(s) origin for Weaviate string - HTTP origin
    CONFIGURATION_STORAGE_URL Service-Discovery for the (etcd) config store. string - URL http://etcd:2379
    CONTEXTIONARY_URL Service-Discovery for the contextionary container string - URL http://contextionary
    ESVECTOR_URL Service-Discovery for the Elasticsearch instance string - URL http://esvector:9200
    ESVECTOR_NUMBER_OF_SHARDS Configure default number of ES shards int 1
    ESVECTOR_AUTO_EXPAND_REPLICAS Wheter ES should auto expand replicas string 1-3
    STANDALONE_MODE Turn on experimental standalone mode string - true/false false
    PERSISTENCE_DATA_PATH Only if STANDALONE_MODE=true: Where should Weaviate Standalone store its data? string - file path /var/lib/weaviate
    AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED Allow users to interact with weaviate without auth string - true/false true
    AUTHENTICATION_OIDC_ENABLED Enable OIDC Auth string - true/false false
    AUTHENTICATION_OIDC_ISSUER OIDC Token Issuer string - URL https://myissuer.com
    AUTHENTICATION_OIDC_CLIENT_ID OIDC Client ID string my-client-id
    AUTHENTICATION_OIDC_USERNAME_CLAIM OIDC Username Claim string email
    AUTHENTICATION_OIDC_GROUPS_CLAIM OIDC Groups Claim string groups
    AUTHORIZATION_ADMINLIST_ENABLED Enable AdminList Authorization mode string - true/false true
    AUTHORIZATION_ADMINLIST_USERS Users with admin permission string - comma-separated list [email protected],[email protected]
    AUTHORIZATION_ADMINLIST_READONLY_USERS Users with read-only permission string - comma-separated list [email protected],[email protected]

    More CRUD capabilities in Standalone-Mode (#1221)
    ๐Ÿ“š If you are running the experimental standalone mode (a preview of some of the features we will introduce in 1.0.0), more CRUD capabilities were added. See #1221 for details and the "Standalone Mode" status page for an up-to-date feature/limitations overview.

    ๐Ÿ›  Fixes

    • โœ‚ Remove broken/unused/impossible endpoints (#1235)
      ๐Ÿš€ Prior to this release, there were some REST endpoints that were never implemented or always failed, due to the action not being possible. They have been cleaned up in this release. This removal of endpoints is considered non-breaking, as they were never usable to begin with. See #1235 for which endpoints were removed.
  • v0.22.15 Changes

    August 28, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.15
    ๐Ÿ‘€ See also: example docker compose files in English, German, Dutch, Italian and Czech.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    Optional Compound Splitting in Contextionary

    Motivation

    Sometimes Weaviate's Contextionary does not understand words which are compounded out of words it would otherwise understand. This impact is far greater in languages that allow for arbritrary compounding (such as Dutch or German) than in languages where compounding is not very common (such as English).

    Effect

    ๐Ÿšš Imagine you import an object of class Post with content This is a thunderstormcloud. The arbitrarily compunded word thunderstormcloud is not present in the Contextionary. So your object's position will be made up of the only words it recognizes: "post", "this" ("is" and "a" are removed as stopwords).

    ๐Ÿ‘€ If you check how this content was vectorized using the _interpration feature, you will see something like the following:

    "\_interpretation": { "source": [{ "concept": "post", "occurrence": 62064610, "weight": 0.3623903691768646 }, { "concept": "this", "occurrence": 932425699, "weight": 0.10000000149011612 }] }
    

    To overcome this limitation the optional Compound Splitting Feature can be enabled in the Contextionary. It will understand the arbitrary compounded word and interpret your object as follows:

    "\_interpretation": { "source": [{ "concept": "post", "occurrence": 62064610, "weight": 0.3623903691768646 }, { "concept": "this", "occurrence": 932425699, "weight": 0.10000000149011612 }, { "concept": "thunderstormcloud (thunderstorm, cloud)", "occurrence": 5756775, "weight": 0.5926488041877747 }] }
    

    Note that the newly found word (made up of the parts thunderstorm and cloud has the highest weight in the vectorization. So this meaning that would have been lost without Compound Splitting can now be recognized.

    Trade-Off Import speed vs Word recognition

    0๏ธโƒฃ Compound Splitting runs an any word that is otherwise not recognized. Depending on your dataset this can lead to a signifcantly longer import time (up to 100% longer). Therefore, you should carefully evaluate whether the higher precision in recognition or the faster import times are more important to your use case. As the benefit is larger in some languages (e.g. Dutch, German) than in others (e.g. English) this feature is turned off by default.

    How to activate

    To turn on compound splitting simply change the environment variable ENABLE_COMPOUND_SPLITTING to true on the contextionary container. For example, on the English language docker-compose files, the variable can be found in this line.

    Classification Perfomance Improvements
    ๐Ÿš€ Prior to this release Classifications (both kNN and contextual) were single threaded and thus not utilizing all available resources. With this release both classification types will use as many threads as CPU cores are available. This can speed up classifications considerably on larger machines.

    ๐Ÿ›  Fixes

    none

  • v0.22.14 Changes

    July 14, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.14
    ๐Ÿ‘€ See also: example docker compose files in English, German, Dutch, Italian and Czech.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    ๐Ÿ‘Œ Improved noise filtering in _nearestNeighbors and _semanticPath (semi-technologies/contextionary#35)
    Prior to this release the features relying on nearest neighbors (namely _nearestNeighbors and _semanticPath) might have contained "noise words", i.e. concatenated words of random words with no easily recognizable meaning. These words are present in the Contextionary training space, but are extremely rare and therefore distributed seemingly randomly. As a result an "ordinary" result might have such a noise word as it's immediate neighbor.

    To combat this noise, a neighbor filtering feature was introduced in the contextionary, which ignores words of the configured bottom percentile - ranked by occurrence in the respective training set. By default this value is set to the bottom 5th percentile. This setting can be overridden. To set another value, e.g. to ignore the bottom 10th percentile, provide the environment variable NEIGHBOR_OCCURRENCE_IGNORE_PERCENTILE=10 to the contextionary container. An example of this variable set correctly can be found here (english version) for example.

    Note that this feature requires at least contextionary version ...-v0.4.15.

    ๐Ÿ›  Fixes

    ๐Ÿ”ง Check if required contextionary container is running and configured at startup (gh-1190)
    ๐Ÿ”ง Prior to this version, it was possible to run a new version of weaviate together with an outdated and incompatible contextionary version. This (invalid) combination would start up fine, but eventually lead to runtime errors when a contextionary method was called that was incompatible with the configured contextionary container.

    ๐Ÿ”ง This fix makes this impossible. If a contextionary version is configured that does not meet the required conditions, Weaviate will no longer start up. The logs will show one of the following options:

    Option 1: Wrong version

    FATA[0001] insufficient contextionary version, cannot start up action=startup_check_contextionary contextionaryVersion=en0.16.0-v0.4.14 requiredMinimumContextionaryVersion=0.4.15
    exit status 1
    

    Option 2: Happy path

    INFO[0001] found a valid contextionary version action=startup_check_contextionary contextionaryVersion=en0.16.0-v0.4.15 requiredMinimumContextionaryVersion=0.4.15
    
  • v0.22.13 Changes

    July 10, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.13
    ๐Ÿ‘€ See also: example docker compose files in English, German, Dutch, Italian and Czech.

    ๐Ÿ’ฅ Breaking Changes

    ๐Ÿ†• New Features

    โž• Add _semanticPath underscore prop to Get{} queries with explore set (#1168)
    The semantic path is the path between a search term and each result. For example if the search term is iphone and the first result is an article about "Microsoft Inc.", then the path would be how to get from the search term to the result, for example ["iphone", "apple", "company", "microsoft"]

    ๐Ÿ— The semantic path can be queried using the _semanticPath GraphQL property alongside the schema-defined properties in GraphQL. Building a semantic path is only possible if an explore: {} parameter is set. As the explore term represents the beginning of the path and each search result represents the end of the path. If no explore: {} parameter is set, the query is just a list query and not a search query, therefore no path can be built. Since explore:{} queries are currently exclusively possible in GraphQL, the _semanticPath is therefore not available in the REST API, either.

    Notes

    1. Due to the relatively high cost of the underlying algorithm, we recommend to limit requests including a _featureProjection in high-load situations where response time matters. Avoid parallel requests including a _featureProjection, so that some threads stay available to serve other, time-critical requests. The maximum supported request limit (to be set through the limit: <int> parameter is 25
    2. The semantic path feature requires a contextionary version of at least ...-v0.4.14 which is used in the provided docker-compose files linked above.

    ๐Ÿ›  Fixes

  • v0.22.12 Changes

    June 26, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.12
    ๐Ÿ‘€ See also: example docker compose files in English, German, Dutch, Italian and Czech.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    ๐Ÿ†• New Underscore Prop for visualization: _featureProjection (#1178, #1139)
    ๐Ÿš€ This release adds a new optional property ("underscore prop") to list results (REST GET /v1/{kinds}, GraphQL Get {}). The feature projection is intended to reduce the dimensionality of the object's vector into something easily suitable for visualizing, such as 2d or 3d. The underlying algorithm is exchangeable, the first algorithm to be provided is t-SNE.

    ๐Ÿ”‹ The feature can be used without any params and tries to pick reasonable defaults. To do so, use the include parameter on REST (GET /v1/{kinds}/?include=_featureProjection) or the_featureProjection { vector }` paramter in GraphQL which appears alongside the schema-defined properties.

    Optional Parameteres

    0๏ธโƒฃ To tweak the feature projection optional paramaters (currently GraphQL-only) can be provided. The values and their defaults are:

    0๏ธโƒฃ | Parameter | Type | Default | Implication | | --- | --- | --- | --- | | dimensions | int | 2 | Target dimensionality, usually 2 or 3 | ๐Ÿ‘ | algorithm | string | tsne | Algorithm to be used, currently supported: tsne | | perplexity | int | min(5, len(results)-1) | The t-SNE perplexity value, must be smaller than the n-1 where n is the number of results to be visualized | | learningRate | int | 25 | The t-SNE learning rate | | iterations | int | 100 | The number of iterations the t-SNE algorithm runs. Higher values lead to more stable results at the cost of a larger response time |

    Limitations and Restrictions

    • There is no request size limit (other than the global 10,000 items request limit) which can be used on a _featureProjection query. However, due to the O(n2) complexity of the t-SNE algorithm, large requests size have an exponential effect on the response time. We recommend to keep the request size at or below 100 items, as we have noticed drastic increases in response time thereafter.
    • Feature Projection happens in real-time, per query. The dimensions returned have no meaning across queries.
    • Currently only root elements (not resolved cross-references) are taken into consideration for the featureProjection.
    • Due to the relatively high cost of the underlying algorithm, we recommend to limit requests including a _featureProjection in high-load situations where response time matters. Avoid parallel requests including a _featureProjection, so that some threads stay available to serve other, time-critical requests.

    Example

    The screenshot below shows a visualization done on a subset of the 20 newsgroup dataset with the article's main category used as label. The chart was created in Python using matplotlib.pyplot's scatter feature.

    image

    ๐Ÿ›  Fixes

    none

  • v0.22.11 Changes

    June 24, 2020

    ๐Ÿณ Docker image/tag: semitechnologies/weaviate:0.22.11
    ๐Ÿ‘€ See also: example docker compose files in English, German, Dutch, Italian and Czech.

    ๐Ÿ’ฅ Breaking Changes

    none

    ๐Ÿ†• New Features

    โž• Add _nearestNeighbors underscore prop to REST and GraphQL API (#1169)
    Display information about the neighboring concepts of an object in a single (GET /v1/{kind}/{id}) or list (GET /v1/{kinds}) REST response by setting the ?include=_nearestNeighbors property. The same can be achieved in a GraphQL Get {} query by requesting the _nearestNeighbors{} prop alongside the schema-defined props.

    Note: Displaying nearest neighbors leads to n additional searches, where n is the number of search results. Use carefully in high-load situations or scale up Weaviate accordingly.

    ๐Ÿ›  Fixes

    • Missing classification distances in meta/_classification responses (#1176)
      ๐Ÿš€ Release 0.22.8 introduced an issue where the kNN-classification distances were sometimes not shown when the ?meta=true (deprecated) or the ?include=_classification flags were set. This release fixes this issue.