Popularity
4.6
Growing
Activity
8.2
-
725
20
49

Programming language: C
License: GNU General Public License v3.0 only
Tags: Search Engines    
Latest version: v2.5.0

sist2 alternatives and similar software solutions

Based on the "Search Engines" category.
Alternatively, view sist2 alternatives based on common mentions on social networks and blogs.

  • MeiliSearch

    A lightning-fast search API that fits effortlessly into your apps, websites, and workflow
  • Searx

    9.3 7.7 L2 sist2 VS Searx
    Privacy-respecting metasearch engine
  • The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
    Promo
  • Typesense

    Open Source alternative to Algolia + Pinecone and an Easier-to-Use alternative to ElasticSearch ⚡ 🔍 ✨ Fast, typo tolerant, in-memory fuzzy Search Engine for building delightful search experiences
  • Yacy

    7.3 8.8 L1 sist2 VS Yacy
    Distributed Peer-to-Peer Web Search Engine and Intranet Search Appliance
  • Ambar

    :mag: Ambar: Document Search Engine
  • Gigablast

    Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.
  • Seeks

    3.5 0.0 L1 sist2 VS Seeks
    Seeks is a decentralized p2p websearch and collaborative tool.
  • multiSearchHome

    :mag_right: Local standalone html homepage to search in 175 search engine (duckduckgo, youtube, twitter, wikipedia, etc..) // FR___: Page d'accueil html autonome, pour chercher dans 175 moteurs de recherche.

Do you think we are missing an alternative of sist2 or a related project?

Add another 'Search Engines' Software solution

README

GitHub CodeFactor Development snapshots

Demo: sist2.simon987.net

sist2

sist2 (Simple incremental search tool)

Warning: sist2 is in early development

[search panel](docs/sist2.png)

Features

  • Fast, low memory usage, multi-threaded
  • Mobile-friendly Web interface
  • Portable (all its features are packaged in a single executable)
  • Extracts text and metadata from common file types *
  • Generates thumbnails *
  • Incremental scanning
  • Manual tagging from the UI and automatic tagging based on file attributes via [user scripts](docs/scripting.md)
  • Recursive scan inside archive files **
  • OCR support with tesseract ***
  • Stats page & disk utilisation visualization

* See format support
** See Archive files
*** See OCR

[stats](docs/stats.png)

Getting Started

  1. Have an Elasticsearch (>= 6.8.X, ideally >=7.14.0) instance running
    1. Download from official website
    2. (or) Run using docker: bash docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.14.0
    3. (or) Run using docker-compose: yaml elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.14.0 environment: - discovery.type=single-node - "ES_JAVA_OPTS=-Xms1G -Xmx2G"
  2. Download sist2 executable

    1. Download the latest sist2 release. Select the file corresponding to your CPU architecture and mark the binary as executable with chmod +x *
    2. (or) Download a development snapshot (Not recommended!)
    3. (or) docker pull simon987/sist2:2.12.1-x64-linux
  3. See [Usage guide](docs/USAGE.md)

* Windows users: sist2 runs under WSL

Example usage

See [Usage guide](docs/USAGE.md) for more details

  1. Scan a directory: sist2 scan ~/Documents -o ./docs_idx
  2. Push index to Elasticsearch: sist2 index ./docs_idx
  3. Start web interface: sist2 web ./docs_idx

Format support

File type Library Content Thumbnail Metadata
pdf,xps,fb2,epub MuPDF text+ocr yes author, title
cbz,cbr libscan - yes -
audio/* ffmpeg - yes ID3 tags
video/* ffmpeg - yes title, comment, artist
image/* ffmpeg ocr yes Common EXIF tags, GPS tags
raw, rw2, dng, cr2, crw, dcr, k25, kdc, mrw, pef, xf3, arw, sr2, srf, erf LibRaw no yes Common EXIF tags, GPS tags
ttf,ttc,cff,woff,fnt,otf Freetype2 - yes, bmp Name & style
text/plain libscan yes no -
html, xml libscan yes no -
tar, zip, rar, 7z, ar ... Libarchive yes* - no
docx, xlsx, pptx libscan yes if embedded creator, modified_by, title
doc (MS Word 97-2003) antiword yes yes author, title
mobi, azw, azw3 libmobi yes no author, title
wpd (WordPerfect) libwpd yes no planned
json, jsonl, ndjson libscan yes - -

* See Archive files

Archive files

sist2 will scan files stored into archive files (zip, tar, 7z...) as if they were directly in the file system. Recursive (archives inside archives) scan is also supported.

Limitations:

  • Support for parsing media files with formats that require seek (e.g. .gif, .mp4 w/ fragmented metadata etc.) is limitted (see --mem-buffer option)
  • Archive files are scanned sequentially, by a single thread. On systems where sist2 is not I/O bound, scans might be faster when larger archives are split into smaller parts.

OCR

You can enable OCR support for ebook (pdf,xps,fb2,epub) or image file types with the --ocr-lang <lang> option in combination with --ocr-images and/or --ocr-ebooks. Download the language data files with your package manager (apt install tesseract-ocr-eng) or directly from Github.

The simon987/sist2 image comes with common languages (hin, jpn, eng, fra, rus, spa) pre-installed.

You can use the + separator to specify multiple languages. The language name must be identical to the *.traineddata file installed on your system (use chi_sim rather than chi-sim).

Examples:

sist2 scan --ocr-ebooks --ocr-lang jpn ~/Books/Manga/
sist2 scan --ocr-images --ocr-lang eng ~/Images/Screenshots/
sist2 scan --ocr-ebooks --ocr-images --ocr-lang eng+chi_sim ~/Chinese-Bilingual/

Build from source

You can compile sist2 by yourself if you don't want to use the pre-compiled binaries

With docker (recommended)

git clone --recursive https://github.com/simon987/sist2/
cd sist2
docker build . -f ./Dockerfile -t my-sist2-image
docker run --rm --entrypoint cat my-sist2-image /root/sist2 > sist2-x64-linux

On a linux computer

  1. Install compile-time dependencies
   apt install gcc g++ python3 yasm ragel automake autotools-dev wget libtool libssl-dev curl zip unzip tar xorg-dev libglu1-mesa-dev libxcursor-dev libxml2-dev libxinerama-dev gettext nasm git
  1. Apply vcpkg patches, as per sist2-build Dockerfile

  2. Install vcpkg dependencies

    vcpkg install curl[core,openssl]
    vcpkg install lmdb cjson glib brotli libarchive[core,bzip2,libxml2,lz4,lzma,lzo] pthread tesseract libxml2 libmupdf gtest mongoose libmagic libraw jasper lcms gumbo
    
  3. Build

    git clone --recursive https://github.com/simon987/sist2/
    cmake -DSIST_DEBUG=off -DCMAKE_TOOLCHAIN_FILE=<VCPKG_ROOT>/scripts/buildsystems/vcpkg.cmake .
    make
    


*Note that all licence references and agreements mentioned in the sist2 README section above are relevant to that project's source code only.