This is an automated archive.

The original was posted on /r/opensource by /u/BetaCygni on 2023-08-07 08:42:07+00:00.

Hi everyone! Over the past weeks I’ve been working on DawnSearch, an open source distributed web search engine, using semantic (vector) search. It is written in Rust and uses some amazing libraries, like rust-bert and USearch.

Web search: dawnsearch.org Source code: github.com/dawn-search/dawnsearch

The goal of this is to get some form of open source web search going. The recent advances in AI embedding and vector search may have made it possible to do this in a distributed way, so no single person or organization will have to run a massive datacenter like the current search giants.

I feel what it is so far is functional enough for other people to look at. Let me know what you think, both about the concept, the current proof of concept, and if you’re up for it the code itself.